-- Logs begin at Thu 2022-08-25 01:52:14 UTC, end at Thu 2022-08-25 03:46:56 UTC. -- Aug 25 02:59:27.708053 dv-cinder-851640-5-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 25 02:59:27.717453 dv-cinder-851640-5-iscsi devstack@keystone.service[81983]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 25 02:59:27.719527 dv-cinder-851640-5-iscsi devstack@keystone.service[81983]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Aug 25 02:59:27 2022] *** Aug 25 02:59:27.719527 dv-cinder-851640-5-iscsi devstack@keystone.service[81983]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 25 02:59:27.719527 dv-cinder-851640-5-iscsi devstack@keystone.service[81983]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 25 02:59:27.719527 dv-cinder-851640-5-iscsi devstack@keystone.service[81983]: nodename: dv-cinder-851640-5-iscsi Aug 25 02:59:27.719527 dv-cinder-851640-5-iscsi devstack@keystone.service[81983]: machine: x86_64 Aug 25 02:59:27.719527 dv-cinder-851640-5-iscsi devstack@keystone.service[81983]: clock source: unix Aug 25 02:59:27.719527 dv-cinder-851640-5-iscsi devstack@keystone.service[81983]: pcre jit disabled Aug 25 02:59:27.719527 dv-cinder-851640-5-iscsi devstack@keystone.service[81983]: detected number of CPU cores: 4 Aug 25 02:59:27.719527 dv-cinder-851640-5-iscsi devstack@keystone.service[81983]: current working directory: / Aug 25 02:59:27.719527 dv-cinder-851640-5-iscsi devstack@keystone.service[81983]: detected binary path: /usr/bin/uwsgi-core Aug 25 02:59:27.719527 dv-cinder-851640-5-iscsi devstack@keystone.service[81983]: your processes number limit is 47680 Aug 25 02:59:27.719527 dv-cinder-851640-5-iscsi devstack@keystone.service[81983]: your memory page size is 4096 bytes Aug 25 02:59:27.720345 dv-cinder-851640-5-iscsi devstack@keystone.service[81983]: detected max file descriptor number: 2048 Aug 25 02:59:27.720345 dv-cinder-851640-5-iscsi devstack@keystone.service[81983]: lock engine: pthread robust mutexes Aug 25 02:59:27.720345 dv-cinder-851640-5-iscsi devstack@keystone.service[81983]: thunder lock: enabled Aug 25 02:59:27.720345 dv-cinder-851640-5-iscsi devstack@keystone.service[81983]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 25 02:59:27.720345 dv-cinder-851640-5-iscsi devstack@keystone.service[81983]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 25 02:59:27.759501 dv-cinder-851640-5-iscsi devstack@keystone.service[81983]: Python main interpreter initialized at 0x557fe564c9c0 Aug 25 02:59:27.759501 dv-cinder-851640-5-iscsi devstack@keystone.service[81983]: python threads support enabled Aug 25 02:59:27.759501 dv-cinder-851640-5-iscsi devstack@keystone.service[81983]: your server socket listen backlog is limited to 100 connections Aug 25 02:59:27.759501 dv-cinder-851640-5-iscsi devstack@keystone.service[81983]: your mercy for graceful operations on workers is 90 seconds Aug 25 02:59:27.759856 dv-cinder-851640-5-iscsi devstack@keystone.service[81983]: mapped 403077 bytes (393 KB) for 2 cores Aug 25 02:59:27.759933 dv-cinder-851640-5-iscsi devstack@keystone.service[81983]: *** Operational MODE: preforking *** Aug 25 02:59:27.760177 dv-cinder-851640-5-iscsi devstack@keystone.service[81983]: *** uWSGI is running in multiple interpreter mode *** Aug 25 02:59:27.760177 dv-cinder-851640-5-iscsi devstack@keystone.service[81983]: spawned uWSGI master process (pid: 81983) Aug 25 02:59:27.760378 dv-cinder-851640-5-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 25 02:59:27.762824 dv-cinder-851640-5-iscsi devstack@keystone.service[81983]: spawned uWSGI worker 1 (pid: 81994, cores: 1) Aug 25 02:59:27.762824 dv-cinder-851640-5-iscsi devstack@keystone.service[81983]: spawned uWSGI worker 2 (pid: 81995, cores: 1) Aug 25 02:59:27.762824 dv-cinder-851640-5-iscsi devstack@keystone.service[81983]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 25 02:59:32.111416 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 25 02:59:32.111416 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: from cryptography.utils import int_from_bytes Aug 25 02:59:32.112208 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 25 02:59:32.112208 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: from cryptography.utils import int_from_bytes Aug 25 02:59:32.112856 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 25 02:59:32.112856 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: from cryptography.utils import int_from_bytes Aug 25 02:59:32.113304 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 25 02:59:32.113304 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: from cryptography.utils import int_from_bytes Aug 25 02:59:32.163879 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 25 02:59:32.164057 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 25 02:59:32.164280 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] command line args: [] {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 25 02:59:32.164516 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 25 02:59:32.164695 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] ================================================================================ {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 25 02:59:32.164950 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] admin_token = **** {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.165198 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] config_dir = [] {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.165388 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.165572 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] config_source = [] {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.165755 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] debug = True {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.165962 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.166343 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 25 02:59:32.166441 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.166510 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.166572 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 25 02:59:32.166720 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] command line args: [] {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 25 02:59:32.166804 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.167000 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 25 02:59:32.167077 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.167181 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ================================================================================ {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 25 02:59:32.167255 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] list_limit = None {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.167394 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] log_config_append = None {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.167470 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] admin_token = **** {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.167584 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.167680 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] config_dir = [] {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.167772 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] log_dir = None {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.167863 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.167975 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] log_file = None {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.168069 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] config_source = [] {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.168173 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.168284 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] debug = True {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.168400 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.168560 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.168684 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.168770 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.168885 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.168965 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.169088 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.169169 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.169169 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.169359 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.169433 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.169563 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] list_limit = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.169677 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] log_config_append = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.169797 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.169882 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.169960 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.170031 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] log_dir = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.170143 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.170223 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] log_file = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.170357 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.170472 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.170548 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.170618 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.170759 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.170882 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.170994 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.171113 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.171235 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.171346 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.171459 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.171626 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.171752 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.58/identity {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.171869 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.171987 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] publish_errors = False {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.172115 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.172228 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.172339 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.172457 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.172568 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.172677 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.172794 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.172917 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.173035 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.173163 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.173303 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] standard_threads = False {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.173417 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.173534 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.173695 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.173810 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.173810 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.174039 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] use_journal = False {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.174157 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.58/identity {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.174310 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] use_json = False {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.174425 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] publish_errors = False {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.174536 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] use_stderr = False {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.174648 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.174771 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] use_syslog = False {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.174884 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.174995 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.175106 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.175222 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.175338 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.175466 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.175579 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.175697 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.175808 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] standard_threads = False {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.175919 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.176031 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.176151 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.176263 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.176373 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.176490 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.176490 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] use_journal = False {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.176704 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.176894 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] use_json = False {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.177009 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.177122 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] use_stderr = False {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.177122 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] use_syslog = False {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.177336 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.177448 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:59:32.177571 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.177690 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.177802 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.177912 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.178023 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.178143 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.178261 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.178399 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.178995 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.179113 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.179245 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.179464 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.179588 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.179708 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.179820 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.179931 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.180050 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.180171 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.180295 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.180408 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.180526 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.180643 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.180755 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.180878 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.180996 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.181108 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.181226 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.181344 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.181457 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.181576 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.181694 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.181812 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.181924 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.182036 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.182152 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.182272 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.182421 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.182541 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.182659 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.182771 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.182882 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.183068 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.183188 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.183301 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.183416 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.183416 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.183624 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.183747 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.183860 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.183972 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.184089 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.184201 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.184313 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.184313 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.184522 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.184634 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.184745 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.184865 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.185028 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.185203 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.185314 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.185314 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.185491 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.185597 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.185710 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.185832 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.185928 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.186022 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.186117 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.186218 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.188631 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.188759 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.188841 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.188918 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.188989 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.189087 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.189204 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.189314 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.189420 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.189516 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.189516 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.189693 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.189795 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.189892 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.189986 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.190080 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.190175 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.190258 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.190418 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.190507 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.190580 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.190656 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.190727 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.190849 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.190940 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.191027 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.191088 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.191202 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.191275 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.191341 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.191455 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.191537 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.191653 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.191734 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.191849 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.191932 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.192005 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.192125 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.192209 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.192281 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.192394 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.192467 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.192579 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.192653 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.192774 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.192850 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.192922 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.193035 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.193108 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.193229 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.193298 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.193368 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.193480 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.193554 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.193666 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.193748 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.193826 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.193897 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.194018 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.194242 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.194933 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.195015 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.195089 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.195150 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.195207 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.195264 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.195320 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.195382 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.195516 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.195583 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.195641 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.195697 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.195761 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.195831 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.195911 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.195978 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.196048 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.196048 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.196048 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.196281 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.196530 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.196723 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.196985 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.197242 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.197404 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.197482 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.197667 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.197758 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.197942 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.198025 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.198184 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.198261 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.198450 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.198544 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.198691 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.198768 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.198925 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.199000 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.199161 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.199246 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.199397 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.199476 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.199634 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.199710 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.199866 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.199952 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.200106 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.200184 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.200343 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.200418 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.200594 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.200680 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.200834 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.200912 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.201075 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.201151 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.201318 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.201419 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.201825 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.201916 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.202063 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] auth.external = None {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.202139 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.202315 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.202408 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.202560 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.202639 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.202795 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.202872 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.203030 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.203116 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] auth.password = None {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.203265 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.203344 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] auth.token = None {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.203504 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.203579 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.203735 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.203822 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.203979 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.204075 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.204209 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.204292 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.204458 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.204538 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.204709 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.204795 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.204945 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.205033 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.205191 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] credential.caching = True {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.205318 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.205442 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.205581 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.205703 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] auth.external = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.205818 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.205931 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.206051 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.206164 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.206305 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.206425 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.206539 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.206651 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.206774 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] auth.password = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.206900 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] auth.token = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.207014 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.207127 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.207239 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.207352 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.207465 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.207587 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.207701 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.207814 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.207926 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.208038 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.208157 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.208280 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.208394 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.208508 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.208621 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.208734 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] credential.caching = True {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.208847 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.208971 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.209095 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] federation.caching = True {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.209210 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.209324 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.209436 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.209550 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.209750 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.209859 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.209956 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.210052 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.210190 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.210352 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.210483 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.210599 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.210716 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.210830 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.210943 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.211086 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.211214 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.211330 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.211444 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.211557 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.211672 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.211785 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.211906 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] identity.caching = True {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.212021 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.212134 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.212248 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.212361 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.212474 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.212596 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.212712 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] federation.caching = True {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.212825 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.212938 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.213051 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.213163 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.213284 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.213399 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.213511 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.213623 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.213745 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.213914 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.214061 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.214168 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.214266 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.214399 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.214502 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.214599 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.217241 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.217365 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.217442 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.217526 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.217605 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.217672 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.217751 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.217824 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] identity.caching = True {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.217894 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.217964 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.218034 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.218112 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.218184 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.218254 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.218358 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.218431 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.218502 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.218585 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.218659 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.218730 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.218795 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.218854 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.218917 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.218996 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.219073 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.219145 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.219220 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.219292 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.219362 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.219440 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.219513 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.219583 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.219654 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.219724 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.219793 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.219870 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.219943 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.220021 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.220099 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.220170 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.220241 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.220319 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.220391 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.220461 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.220531 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.220601 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.220671 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.220771 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.220849 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.220920 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.220979 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.221044 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.221113 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.221192 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.221264 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.221335 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.221406 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.221476 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.221592 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.221678 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.221752 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.221822 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.221892 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.221961 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.222031 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.222134 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.222270 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.222396 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.222533 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.222613 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.222782 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.222870 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.223024 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.223104 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.223263 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.223373 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.223505 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.223617 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.223740 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] ldap.user = None {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.223851 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.223984 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.224090 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.224245 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.224332 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.224482 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.224560 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.224730 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.224808 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.224979 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.225120 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.225230 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.225309 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.225496 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.225573 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.225730 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.225815 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.225965 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.226044 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.226206 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.226301 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.226481 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.226572 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.226714 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.226808 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.226976 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.227089 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.227204 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.227327 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.227444 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.227558 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.user = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.227670 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.227784 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.227918 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.228073 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.228192 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.228304 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.228426 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.228540 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.228663 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.228777 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.228890 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.229003 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.229125 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.229239 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.229365 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.229482 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.229595 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.229708 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.229826 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.229940 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.230062 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.230177 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.230349 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.230496 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.230614 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.230727 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.230854 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.230969 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.231082 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.231195 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.231308 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.231421 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.231542 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.231658 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.231771 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.231884 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.231997 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.232110 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.232298 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.232414 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.232528 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.232642 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.232755 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] resource.caching = True {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.232868 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.233003 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.233103 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.233199 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.233295 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.233390 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.233492 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.233598 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.233696 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.233792 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.233889 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.233984 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.234079 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.234184 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.234302 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.234412 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.234510 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.234615 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.234712 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.238514 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] role.caching = True {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.238625 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.238705 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] role.driver = sql {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.238822 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.238898 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.238969 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.239053 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.239126 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.239196 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.239267 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.239337 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.239407 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] resource.caching = True {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.239485 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.239558 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.239629 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.239699 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.239770 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.239841 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.239919 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.239992 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.240062 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.240133 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.240203 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.240273 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.240352 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.240424 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.240495 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.240604 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.240682 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.240797 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.240881 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.241440 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] role.caching = True {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.241440 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] role.driver = sql {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.241575 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.241645 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.241715 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.241794 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.241866 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.241937 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.242007 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.242077 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.242155 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.242228 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.242314 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.242388 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.242458 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.242528 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.242610 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.242683 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.242754 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.242824 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.242894 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.242964 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.243042 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.243042 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.243218 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.243293 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.243363 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.243434 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.243513 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.243586 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.243678 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.243750 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.243810 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.243925 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.244010 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.244083 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.244154 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.244231 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.244303 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.244374 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] token.caching = True {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.244374 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.244510 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.244580 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.244650 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.244719 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.244788 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.244866 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.244937 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.245007 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.245077 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.245146 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.245277 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.245277 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.245421 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.245499 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.245571 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.245641 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.245711 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.245861 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.245987 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.246107 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.246185 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.246385 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.246480 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.246635 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.246714 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.246899 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.246976 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.247131 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.247226 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.247366 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] token.caching = True {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.247453 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.247604 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.247683 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.247853 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.247932 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] database.connection = **** {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.248093 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.248180 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.248343 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.248421 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.248585 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.248664 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.248820 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.248909 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.249056 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.249133 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.249296 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.249374 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.249535 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.249635 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.249775 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.249854 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.250021 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.250100 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.250260 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.250398 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.250531 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.250611 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.250770 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.250848 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.251014 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.251105 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.251254 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.251329 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.251482 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.251562 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.251714 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.251792 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] database.connection = **** {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.251957 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.252038 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.252195 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.252274 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.252369 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81995) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 25 02:59:32.252448 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.252662 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.252898 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.253136 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.253373 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.253612 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.253856 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.254093 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.254363 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.254606 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.254854 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.255091 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.255336 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.255568 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.255808 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.256045 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:59:32.256215 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 25 02:59:32.282305 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81995) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:59:32.282557 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81995) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:59:32.282790 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81995) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:59:32.285463 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81994) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:59:32.285699 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81994) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:59:32.285930 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81994) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:59:32.291945 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81995) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:59:32.294932 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81994) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:59:32.320520 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81995) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:59:32.320754 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81995) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:59:32.324834 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81994) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:59:32.325066 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81994) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:59:32.392365 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81995) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:59:32.392570 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81994) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:59:32.441024 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81995) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:59:32.441264 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81995) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:59:32.441916 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81994) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:59:32.442155 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81994) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:59:32.602386 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81995) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:59:32.602621 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81995) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:59:32.602840 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81995) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:59:32.603009 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81994) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:59:32.603098 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81995) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:59:32.603257 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81995) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:59:32.603372 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81994) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:59:32.603485 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81995) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:59:32.603758 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81994) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:59:32.604140 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81994) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:59:32.604514 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81994) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:59:32.604897 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81994) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:59:32.631992 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81995) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:59:32.632151 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81994) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:59:32.632223 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81995) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:59:32.632392 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81994) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:59:32.632473 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81995) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:59:32.632621 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81994) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:59:32.632706 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81995) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:59:32.632843 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81994) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:59:32.632935 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81995) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:59:32.633060 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81994) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:59:32.633124 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81995) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:59:32.633728 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81994) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:59:32.633794 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81995) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:59:32.633867 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81994) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:59:32.645692 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81995) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:59:32.645801 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81994) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:59:32.657868 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81995) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:59:32.658105 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81995) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:59:32.658217 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81994) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:59:32.658340 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81994) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:59:32.658418 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81995) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:59:32.658563 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81994) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:59:32.658660 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81995) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:59:32.658784 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81995) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:59:32.658955 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81994) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:59:32.659097 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81994) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:59:32.659231 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81995) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:59:32.659362 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81994) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:59:32.659493 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81995) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:59:32.659587 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81994) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:59:32.676428 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81995) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.676643 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81995) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 25 02:59:32.676945 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.677034 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81995) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.677160 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 25 02:59:32.677249 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81995) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 25 02:59:32.677444 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81995) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.677527 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.677688 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 25 02:59:32.677772 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81995) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.677963 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.678047 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81995) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.678325 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.678581 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81995) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.679198 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.679597 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.688528 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81995) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.688812 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81995) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.688987 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.689094 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81995) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.689270 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.690975 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81995) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.691076 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.691076 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.696448 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81995) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 25 02:59:32.696948 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81994) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 25 02:59:32.699377 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81995) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 25 02:59:32.700480 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81994) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 25 02:59:32.700619 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81995) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.700774 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.700905 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81995) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.701051 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.701185 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81995) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.701346 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.701465 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81995) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.701956 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.702043 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81995) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.702229 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.702337 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81995) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.702530 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.702613 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81995) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.702812 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.702936 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81995) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.703203 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.703286 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81995) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.703485 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81995) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.703566 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.703757 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.727367 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81995) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.727708 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.728848 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81995) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 25 02:59:32.729141 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81995) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.729224 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81994) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 25 02:59:32.729459 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.733847 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81995) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 25 02:59:32.734139 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81995) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.734225 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81994) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 25 02:59:32.734460 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81995) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.734560 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.734771 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.740863 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81995) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 25 02:59:32.741174 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81994) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 25 02:59:32.741260 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81995) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.741465 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.745322 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81995) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 25 02:59:32.745579 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81994) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 25 02:59:32.745664 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81995) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.745874 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.745958 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81995) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.746163 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.746249 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81995) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.746471 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81995) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.746555 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.746750 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.746845 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81995) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.747026 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81995) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.747108 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.747297 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.747378 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81995) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.747571 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.764569 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81995) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.764795 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.764881 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81995) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.765081 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.768760 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81995) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 25 02:59:32.768986 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81994) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 25 02:59:32.772030 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81995) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.775909 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.776021 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81995) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.776108 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.777069 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81995) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 25 02:59:32.777296 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81994) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 25 02:59:32.780240 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81995) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 25 02:59:32.780454 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81994) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 25 02:59:32.783516 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81995) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.783772 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.783860 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81995) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.784083 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.784167 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81995) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.784373 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81995) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.784465 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.784669 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.784752 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81995) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.784952 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.785045 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81995) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.785247 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.801386 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81995) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 25 02:59:32.801684 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81995) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.801904 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81994) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 25 02:59:32.802003 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81995) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.802210 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.802337 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81995) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.802547 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.802818 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.809671 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81995) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.810217 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.811238 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81995) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.811520 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81995) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.811823 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.812103 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.814176 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81995) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 25 02:59:32.814498 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81995) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.814750 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81994) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 25 02:59:32.814834 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81995) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.815038 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.815121 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81995) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.815339 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.815425 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81995) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.815637 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.815922 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.827555 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81995) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 25 02:59:32.827844 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81995) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.828133 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81995) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.828269 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81994) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 25 02:59:32.828425 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81995) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.828559 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.828716 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81995) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.828848 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.829001 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81995) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.829136 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.829298 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81995) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.829424 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.829705 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.830006 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.844832 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81995) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 25 02:59:32.845556 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81994) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 25 02:59:32.847904 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81995) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 25 02:59:32.850391 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81994) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 25 02:59:32.850850 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81995) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.851589 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.852309 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81995) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.853032 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.853801 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81995) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 25 02:59:32.854092 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81995) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.854407 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81995) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.854550 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81994) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 25 02:59:32.854839 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.855126 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.860951 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81995) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.861647 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.862456 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81995) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 25 02:59:32.863212 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81994) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 25 02:59:32.865347 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81995) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.865647 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81995) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.866453 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81995) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.866667 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.866876 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81995) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.866987 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.867256 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.867536 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.874774 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81995) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 25 02:59:32.875071 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81995) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.884805 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81995) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.884887 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81994) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 25 02:59:32.884887 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.884887 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.885043 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81995) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 25 02:59:32.885043 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81995) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.885043 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81995) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.885043 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81995) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.885276 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81994) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 25 02:59:32.885337 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81995) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.885396 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.885463 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81995) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.885526 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.885584 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81995) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.885643 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.885702 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81995) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.885769 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.885831 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81995) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.885891 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.885950 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81995) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.886010 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.886069 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81995) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.886137 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.886199 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81995) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.886259 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.886346 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81995) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.886408 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.886480 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81995) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.886544 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.886769 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.886997 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.887209 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:59:32.920502 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x557fe564c9c0 pid: 81995 (default app) Aug 25 02:59:32.921709 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x557fe564c9c0 pid: 81994 (default app) Aug 25 02:59:32.923953 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140492a9-282a-4f3c-aee5-8a7d0a8ba0d3 None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:32.924245 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140492a9-282a-4f3c-aee5-8a7d0a8ba0d3 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:32.924523 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140492a9-282a-4f3c-aee5-8a7d0a8ba0d3 None None] PATH_INFO: `/v3` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:32.929267 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1/1] 10.222.0.58 () {50 vars in 791 bytes} [Thu Aug 25 02:59:32 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:59:40.362204 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ea923f-d4d9-4475-b1ab-e2205e0c51a4 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:40.362747 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ea923f-d4d9-4475-b1ab-e2205e0c51a4 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:40.362747 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ea923f-d4d9-4475-b1ab-e2205e0c51a4 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:40.367008 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1/2] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 02:59:40 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 25 02:59:40.372156 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed65adce-55e3-438b-9de5-4a6c5cc1e00a None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:40.372587 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed65adce-55e3-438b-9de5-4a6c5cc1e00a None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:40.372704 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed65adce-55e3-438b-9de5-4a6c5cc1e00a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:40.442798 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: INFO dbcounter [None req-ed65adce-55e3-438b-9de5-4a6c5cc1e00a None None] Registered counter for database keystone Aug 25 02:59:40.461961 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG oslo_db.sqlalchemy.engines [None req-ed65adce-55e3-438b-9de5-4a6c5cc1e00a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81995) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 25 02:59:40.468692 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writer thread running {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 25 02:59:40.653259 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG passlib.handlers.bcrypt [None req-ed65adce-55e3-438b-9de5-4a6c5cc1e00a None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81995) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 25 02:59:40.653677 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG passlib.handlers.bcrypt [None req-ed65adce-55e3-438b-9de5-4a6c5cc1e00a None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81995) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 25 02:59:40.708365 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-ed65adce-55e3-438b-9de5-4a6c5cc1e00a None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:59:40.710000 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-ed65adce-55e3-438b-9de5-4a6c5cc1e00a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:59:40.837545 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 2/3] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 02:59:40 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 466 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 25 02:59:41.287352 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ea923f-d4d9-4475-b1ab-e2205e0c51a4 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:59:41.356833 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: INFO dbcounter [None req-31ea923f-d4d9-4475-b1ab-e2205e0c51a4 None None] Registered counter for database keystone Aug 25 02:59:41.379490 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG oslo_db.sqlalchemy.engines [None req-31ea923f-d4d9-4475-b1ab-e2205e0c51a4 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81994) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 25 02:59:41.385605 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writer thread running {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 25 02:59:41.504557 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ea923f-d4d9-4475-b1ab-e2205e0c51a4 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:59:41.508872 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eac195-587e-47d2-82b6-150ed33dd6a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:59:41.509500 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eac195-587e-47d2-82b6-150ed33dd6a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:41.509784 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eac195-587e-47d2-82b6-150ed33dd6a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:41.510062 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eac195-587e-47d2-82b6-150ed33dd6a0 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:41.518233 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33eac195-587e-47d2-82b6-150ed33dd6a0 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:59:41.577722 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: WARNING keystone.common.rbac_enforcer.enforcer [None req-33eac195-587e-47d2-82b6-150ed33dd6a0 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 25 02:59:41.578007 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33eac195-587e-47d2-82b6-150ed33dd6a0 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:59:41.587846 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 2/4] 10.222.0.58 () {58 vars in 1324 bytes} [Thu Aug 25 02:59:41 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 25 02:59:41.594262 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed65adce-55e3-438b-9de5-4a6c5cc1e00a None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:59:41.609097 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed65adce-55e3-438b-9de5-4a6c5cc1e00a None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:59:41.613526 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74949b3b-aeaf-40dc-acf0-e22b58b15c6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:59:41.614711 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74949b3b-aeaf-40dc-acf0-e22b58b15c6c admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:41.614999 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74949b3b-aeaf-40dc-acf0-e22b58b15c6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:41.615333 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74949b3b-aeaf-40dc-acf0-e22b58b15c6c admin admin] PATH_INFO: `/v3/projects/b981090d110a4dd99327dbd654b9daa9` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:41.616992 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74949b3b-aeaf-40dc-acf0-e22b58b15c6c admin admin] RBAC: Authorizing `identity:get_project(project_id=b981090d110a4dd99327dbd654b9daa9)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:59:41.668894 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: WARNING keystone.common.rbac_enforcer.enforcer [None req-74949b3b-aeaf-40dc-acf0-e22b58b15c6c admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 25 02:59:41.669244 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74949b3b-aeaf-40dc-acf0-e22b58b15c6c admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:59:41.674430 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 3/5] 10.222.0.58 () {56 vars in 1207 bytes} [Thu Aug 25 02:59:41 2022] GET /identity/v3/projects/b981090d110a4dd99327dbd654b9daa9 => generated 344 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:59:43.548302 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79d3b14-7288-4d9a-8a00-10aa96ac72c7 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:43.548889 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79d3b14-7288-4d9a-8a00-10aa96ac72c7 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:43.548889 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79d3b14-7288-4d9a-8a00-10aa96ac72c7 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:43.553019 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 3/6] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 02:59:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 25 02:59:43.559022 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc11cf5-c19a-443a-b352-50f8e65cb198 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:43.559557 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc11cf5-c19a-443a-b352-50f8e65cb198 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:43.559557 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc11cf5-c19a-443a-b352-50f8e65cb198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:43.580681 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-0bc11cf5-c19a-443a-b352-50f8e65cb198 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:59:43.582219 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-0bc11cf5-c19a-443a-b352-50f8e65cb198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:59:43.590251 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 4/7] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 02:59:43 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 25 02:59:43.865603 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79d3b14-7288-4d9a-8a00-10aa96ac72c7 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:59:43.877721 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79d3b14-7288-4d9a-8a00-10aa96ac72c7 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:59:43.881864 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6944db44-9873-41aa-ae05-7798d21ae706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:59:43.882499 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6944db44-9873-41aa-ae05-7798d21ae706 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:43.882782 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6944db44-9873-41aa-ae05-7798d21ae706 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:43.883056 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6944db44-9873-41aa-ae05-7798d21ae706 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:43.891198 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6944db44-9873-41aa-ae05-7798d21ae706 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:59:43.893791 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6944db44-9873-41aa-ae05-7798d21ae706 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:59:43.902984 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 4/8] 10.222.0.58 () {58 vars in 1324 bytes} [Thu Aug 25 02:59:43 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 25 02:59:43.909038 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc11cf5-c19a-443a-b352-50f8e65cb198 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:59:43.921576 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc11cf5-c19a-443a-b352-50f8e65cb198 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:59:43.927289 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b07e55-7ab4-404f-b09d-2c5939820a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:59:43.927934 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b07e55-7ab4-404f-b09d-2c5939820a29 admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:43.928213 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b07e55-7ab4-404f-b09d-2c5939820a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:43.928489 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b07e55-7ab4-404f-b09d-2c5939820a29 admin admin] PATH_INFO: `/v3/users/e8b708e7077c461bb73c230d7b1825fb` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:43.929802 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b07e55-7ab4-404f-b09d-2c5939820a29 admin admin] RBAC: Authorizing `identity:get_user(user_id=e8b708e7077c461bb73c230d7b1825fb)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:59:43.931680 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b07e55-7ab4-404f-b09d-2c5939820a29 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:59:43.933246 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 5/9] 10.222.0.58 () {56 vars in 1198 bytes} [Thu Aug 25 02:59:43 2022] GET /identity/v3/users/e8b708e7077c461bb73c230d7b1825fb => generated 247 bytes in 25 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:59:45.889361 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f57d421-62b2-4cb3-b159-6ef6a78e6d8c None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:45.889859 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f57d421-62b2-4cb3-b159-6ef6a78e6d8c None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:45.889859 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f57d421-62b2-4cb3-b159-6ef6a78e6d8c None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:45.891176 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 5/10] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 02:59:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:59:45.896204 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b843339-cc3c-4c57-8c9f-47074ed0a26c None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:45.896655 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b843339-cc3c-4c57-8c9f-47074ed0a26c None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:45.896888 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b843339-cc3c-4c57-8c9f-47074ed0a26c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:45.914596 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-1b843339-cc3c-4c57-8c9f-47074ed0a26c None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:59:45.915781 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-1b843339-cc3c-4c57-8c9f-47074ed0a26c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:59:45.943233 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6062dc-2852-483a-9085-a44949d9be2f None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:45.943519 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6062dc-2852-483a-9085-a44949d9be2f None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:45.944372 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6062dc-2852-483a-9085-a44949d9be2f None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:45.945695 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 6/11] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 02:59:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 25 02:59:45.951115 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46ae804-6630-4d07-b74a-c916e5422af6 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:45.951399 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46ae804-6630-4d07-b74a-c916e5422af6 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:45.952163 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46ae804-6630-4d07-b74a-c916e5422af6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:45.993526 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG passlib.handlers.bcrypt [None req-d46ae804-6630-4d07-b74a-c916e5422af6 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81994) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 25 02:59:45.993939 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG passlib.handlers.bcrypt [None req-d46ae804-6630-4d07-b74a-c916e5422af6 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81994) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 25 02:59:45.996820 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 6/12] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 02:59:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 02:59:46.039820 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-d46ae804-6630-4d07-b74a-c916e5422af6 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:59:46.041395 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-d46ae804-6630-4d07-b74a-c916e5422af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:59:46.137330 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 7/13] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 02:59:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 25 02:59:46.270529 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b843339-cc3c-4c57-8c9f-47074ed0a26c None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:59:46.296181 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:59:46.379219 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b843339-cc3c-4c57-8c9f-47074ed0a26c None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:59:46.412304 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46ae804-6630-4d07-b74a-c916e5422af6 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:59:46.413523 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0472bc-c94b-41dd-b1ba-425193ba8caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:59:46.414173 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0472bc-c94b-41dd-b1ba-425193ba8caa None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:46.414482 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0472bc-c94b-41dd-b1ba-425193ba8caa None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:46.414761 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0472bc-c94b-41dd-b1ba-425193ba8caa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:46.425769 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0472bc-c94b-41dd-b1ba-425193ba8caa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:59:46.434567 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-7b0472bc-c94b-41dd-b1ba-425193ba8caa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 02:59:46.434567 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:59:46.434567 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:59:46.434567 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:59:46.434567 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:59:46.434567 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:59:46.434567 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:59:46.434567 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:59:46.434567 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:59:46.434567 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:59:46.434567 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:59:46.434567 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:59:46.434567 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:59:46.434567 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:59:46.434567 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:59:46.434567 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:59:46.435538 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:59:46.435538 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:59:46.435538 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:59:46.435538 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:59:46.435538 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:59:46.435538 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:59:46.435538 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:59:46.435538 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:59:46.435538 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:59:46.435538 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:59:46.435538 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:59:46.435538 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 02:59:46.435538 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:59:46.435538 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:59:46.435538 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:59:46.435538 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:59:46.436660 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:59:46.436660 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:59:46.436660 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:59:46.436660 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:59:46.436660 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:59:46.436660 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:59:46.436660 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:59:46.436660 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:59:46.436660 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:59:46.436660 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:59:46.436660 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 02:59:46.436660 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 02:59:46.436660 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 7/14] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Aug 25 02:59:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:59:46.441324 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0472bc-c94b-41dd-b1ba-425193ba8caa None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:59:46.516349 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46ae804-6630-4d07-b74a-c916e5422af6 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:59:46.518621 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:59:46.523247 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:59:46.540926 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0472bc-c94b-41dd-b1ba-425193ba8caa None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:59:46.552965 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1939b5d9-d623-4bbd-adbb-742dd4aeda71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:59:46.553827 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1939b5d9-d623-4bbd-adbb-742dd4aeda71 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:46.554112 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1939b5d9-d623-4bbd-adbb-742dd4aeda71 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:46.554423 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1939b5d9-d623-4bbd-adbb-742dd4aeda71 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:46.573546 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76da92a4-5bf1-448b-a5d4-02814ea70bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:59:46.574150 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76da92a4-5bf1-448b-a5d4-02814ea70bbe None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:46.574460 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76da92a4-5bf1-448b-a5d4-02814ea70bbe None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:46.574735 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76da92a4-5bf1-448b-a5d4-02814ea70bbe None admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:46.575405 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76da92a4-5bf1-448b-a5d4-02814ea70bbe None admin] RBAC: Adding query filter params (name=admin) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:59:46.575709 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76da92a4-5bf1-448b-a5d4-02814ea70bbe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:59:46.578344 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76da92a4-5bf1-448b-a5d4-02814ea70bbe None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:59:46.590126 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 8/15] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Aug 25 02:59:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:59:46.596312 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76da92a4-5bf1-448b-a5d4-02814ea70bbe None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:59:46.623883 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1939b5d9-d623-4bbd-adbb-742dd4aeda71 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:59:46.625425 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1939b5d9-d623-4bbd-adbb-742dd4aeda71 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:59:46.714815 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76da92a4-5bf1-448b-a5d4-02814ea70bbe None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:59:46.735578 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 8/16] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Aug 25 02:59:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 25 02:59:46.740904 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1939b5d9-d623-4bbd-adbb-742dd4aeda71 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:59:46.743951 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:59:46.750202 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bcd3d5-1a46-493a-836a-32f805275c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:59:46.750915 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bcd3d5-1a46-493a-836a-32f805275c91 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:46.751198 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bcd3d5-1a46-493a-836a-32f805275c91 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:46.751568 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bcd3d5-1a46-493a-836a-32f805275c91 None admin] PATH_INFO: `/v3/users/e8b708e7077c461bb73c230d7b1825fb` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:46.753479 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bcd3d5-1a46-493a-836a-32f805275c91 None admin] RBAC: Authorizing `identity:get_user(user_id=e8b708e7077c461bb73c230d7b1825fb)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:59:46.756260 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bcd3d5-1a46-493a-836a-32f805275c91 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:59:46.759745 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 9/17] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Aug 25 02:59:46 2022] GET /identity/v3/users/e8b708e7077c461bb73c230d7b1825fb => generated 247 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:59:46.765187 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bcd3d5-1a46-493a-836a-32f805275c91 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:59:46.779576 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:59:46.841952 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1939b5d9-d623-4bbd-adbb-742dd4aeda71 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:59:46.862127 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bcd3d5-1a46-493a-836a-32f805275c91 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:59:46.874597 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ea45fb-5f45-478b-b6bd-7123c62b6ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:59:46.875438 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ea45fb-5f45-478b-b6bd-7123c62b6ee6 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:46.875714 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ea45fb-5f45-478b-b6bd-7123c62b6ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:46.875982 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ea45fb-5f45-478b-b6bd-7123c62b6ee6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:46.894219 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ea45fb-5f45-478b-b6bd-7123c62b6ee6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:59:46.895700 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ea45fb-5f45-478b-b6bd-7123c62b6ee6 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:59:46.895950 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d158e9-c388-45e2-b68a-09089329b833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:59:46.896584 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d158e9-c388-45e2-b68a-09089329b833 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:46.897029 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d158e9-c388-45e2-b68a-09089329b833 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:46.897161 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d158e9-c388-45e2-b68a-09089329b833 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:46.898294 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d158e9-c388-45e2-b68a-09089329b833 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:59:46.899740 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d158e9-c388-45e2-b68a-09089329b833 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:59:46.901265 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 10/18] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 02:59:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:59:46.907094 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d158e9-c388-45e2-b68a-09089329b833 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:59:46.913497 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-21ea45fb-5f45-478b-b6bd-7123c62b6ee6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:59:46.913497 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:59:46.913497 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:59:46.913497 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:59:46.913497 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:59:46.913497 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:59:46.913497 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:59:46.913497 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:59:46.913497 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:59:46.913497 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:59:46.913497 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:59:46.913497 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:59:46.913497 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 02:59:46.913497 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 02:59:46.913497 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 02:59:46.913497 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 02:59:46.914406 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:59:46.914406 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:59:46.914406 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:59:46.914406 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:59:46.914406 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:59:46.914406 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:59:46.914406 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:59:46.914406 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 02:59:46.914406 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:59:46.914406 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:59:46.914406 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:59:46.914406 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:59:46.914406 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:59:46.914406 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:59:46.914406 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:59:46.914406 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:59:46.914406 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 02:59:46.915356 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 02:59:46.915356 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:59:46.915356 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 02:59:46.915356 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 9/19] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 02:59:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 02:59:46.920176 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ea45fb-5f45-478b-b6bd-7123c62b6ee6 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:59:46.979261 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:59:46.986683 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:59:47.009058 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d158e9-c388-45e2-b68a-09089329b833 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:59:47.031381 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ea45fb-5f45-478b-b6bd-7123c62b6ee6 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:59:47.045174 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e6871e-48db-4dde-8188-bdd389c2794a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:59:47.045855 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e6871e-48db-4dde-8188-bdd389c2794a None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:47.046139 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e6871e-48db-4dde-8188-bdd389c2794a None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:47.046450 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e6871e-48db-4dde-8188-bdd389c2794a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:47.047193 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e6871e-48db-4dde-8188-bdd389c2794a None admin] RBAC: Adding query filter params (role.id=62f19ea272b447ccaab1576cd9bf4fd9, scope.domain.id=default, user.id=e8b708e7077c461bb73c230d7b1825fb) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:59:47.047502 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e6871e-48db-4dde-8188-bdd389c2794a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:59:47.048947 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e6871e-48db-4dde-8188-bdd389c2794a None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:59:47.055643 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 11/20] 10.222.0.58 () {56 vars in 1322 bytes} [Thu Aug 25 02:59:46 2022] GET /identity/v3/role_assignments?user.id=e8b708e7077c461bb73c230d7b1825fb&scope.domain.id=default&role.id=62f19ea272b447ccaab1576cd9bf4fd9 => generated 233 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:59:47.066952 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744ad3b3-92a7-457c-9d6c-93a376380611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:59:47.067588 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744ad3b3-92a7-457c-9d6c-93a376380611 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:47.067870 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744ad3b3-92a7-457c-9d6c-93a376380611 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:47.068142 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744ad3b3-92a7-457c-9d6c-93a376380611 None admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:47.068826 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744ad3b3-92a7-457c-9d6c-93a376380611 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:59:47.069116 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744ad3b3-92a7-457c-9d6c-93a376380611 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:59:47.070587 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744ad3b3-92a7-457c-9d6c-93a376380611 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:59:47.090658 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 10/21] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Aug 25 02:59:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:59:49.158402 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2473aa01-8f2f-4627-b2ba-9db36b3e65c1 None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:49.158910 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2473aa01-8f2f-4627-b2ba-9db36b3e65c1 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:49.158985 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2473aa01-8f2f-4627-b2ba-9db36b3e65c1 None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:49.160460 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 12/22] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 02:59:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:59:49.165999 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b82d22-9b2b-4113-befc-99cc84ed26bb None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:49.166830 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b82d22-9b2b-4113-befc-99cc84ed26bb None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:49.167303 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b82d22-9b2b-4113-befc-99cc84ed26bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:49.190556 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-f6b82d22-9b2b-4113-befc-99cc84ed26bb None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:59:49.192092 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-f6b82d22-9b2b-4113-befc-99cc84ed26bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:59:49.281768 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 11/23] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 02:59:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 25 02:59:49.337661 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781af469-77e5-4c6a-884e-ee0d1b7f68cd None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:49.338128 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781af469-77e5-4c6a-884e-ee0d1b7f68cd None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:49.338457 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781af469-77e5-4c6a-884e-ee0d1b7f68cd None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:49.339965 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 13/24] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 02:59:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:59:49.347885 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f37b7d-543f-4682-8054-a7af3dcd5c17 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:49.348392 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f37b7d-543f-4682-8054-a7af3dcd5c17 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:49.348864 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f37b7d-543f-4682-8054-a7af3dcd5c17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:49.378110 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-f3f37b7d-543f-4682-8054-a7af3dcd5c17 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:59:49.379660 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-f3f37b7d-543f-4682-8054-a7af3dcd5c17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:59:49.435101 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:59:49.462828 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 12/25] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 02:59:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 02:59:49.530992 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f363b63f-67b1-4d0d-9a91-5d1ca5a6cccb None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:49.531289 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f363b63f-67b1-4d0d-9a91-5d1ca5a6cccb None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:49.531543 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f363b63f-67b1-4d0d-9a91-5d1ca5a6cccb None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:49.533028 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 14/26] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 02:59:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:59:49.550531 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c6829f-86f4-4eb9-b576-12d774fbac4c None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:49.550829 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c6829f-86f4-4eb9-b576-12d774fbac4c None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:49.551115 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c6829f-86f4-4eb9-b576-12d774fbac4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:49.575261 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-d1c6829f-86f4-4eb9-b576-12d774fbac4c None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:59:49.576463 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-d1c6829f-86f4-4eb9-b576-12d774fbac4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:59:49.673854 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 13/27] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 02:59:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 02:59:49.702677 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f363b63f-67b1-4d0d-9a91-5d1ca5a6cccb None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:59:49.867215 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f363b63f-67b1-4d0d-9a91-5d1ca5a6cccb None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:59:49.872475 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c6829f-86f4-4eb9-b576-12d774fbac4c None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:59:49.918993 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddc9c64-5475-4fa8-ab10-e827e1803a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:59:49.919972 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddc9c64-5475-4fa8-ab10-e827e1803a3f None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:49.920304 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddc9c64-5475-4fa8-ab10-e827e1803a3f None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:49.920649 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddc9c64-5475-4fa8-ab10-e827e1803a3f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:49.935703 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddc9c64-5475-4fa8-ab10-e827e1803a3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:59:49.941666 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-7ddc9c64-5475-4fa8-ab10-e827e1803a3f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 02:59:49.941666 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:59:49.941666 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:59:49.941666 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:59:49.941666 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:59:49.941666 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:59:49.941666 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:59:49.941666 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:59:49.941666 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:59:49.941666 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:59:49.941666 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:59:49.941666 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:59:49.941666 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:59:49.941666 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:59:49.941666 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:59:49.941666 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:59:49.942500 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:59:49.942500 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:59:49.942500 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:59:49.942500 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:59:49.942500 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:59:49.942500 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:59:49.942500 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:59:49.942500 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:59:49.942500 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:59:49.942500 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:59:49.942500 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:59:49.942500 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 02:59:49.942500 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:59:49.942500 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:59:49.942500 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:59:49.942500 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:59:49.943250 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:59:49.943250 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:59:49.943250 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:59:49.943250 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:59:49.943250 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:59:49.943250 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:59:49.943250 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:59:49.943250 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:59:49.943250 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:59:49.943250 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:59:49.943250 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 02:59:49.943250 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 02:59:49.943250 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 15/28] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Aug 25 02:59:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:59:49.950505 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddc9c64-5475-4fa8-ab10-e827e1803a3f None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:59:49.962631 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:59:50.088162 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddc9c64-5475-4fa8-ab10-e827e1803a3f None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:59:50.089631 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c6829f-86f4-4eb9-b576-12d774fbac4c None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:59:50.106638 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:59:50.135456 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7749fc78-6daa-4aa3-8cf6-96156ce6b07c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:59:50.136073 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7749fc78-6daa-4aa3-8cf6-96156ce6b07c None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:50.136354 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7749fc78-6daa-4aa3-8cf6-96156ce6b07c None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:50.136627 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7749fc78-6daa-4aa3-8cf6-96156ce6b07c None admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:50.137304 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7749fc78-6daa-4aa3-8cf6-96156ce6b07c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:59:50.137606 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7749fc78-6daa-4aa3-8cf6-96156ce6b07c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:59:50.139400 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7749fc78-6daa-4aa3-8cf6-96156ce6b07c None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:59:50.157103 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1df9631-167d-47eb-8b0e-a1f6cd48b66b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:59:50.157758 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1df9631-167d-47eb-8b0e-a1f6cd48b66b None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:50.158034 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1df9631-167d-47eb-8b0e-a1f6cd48b66b None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:50.158326 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1df9631-167d-47eb-8b0e-a1f6cd48b66b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:50.160945 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 16/29] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Aug 25 02:59:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:59:50.161937 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dbed47-4e4c-4597-9e9e-11c73c367881 None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:50.162223 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dbed47-4e4c-4597-9e9e-11c73c367881 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:50.162531 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dbed47-4e4c-4597-9e9e-11c73c367881 None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:50.163717 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 17/30] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 02:59:50 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:59:50.172957 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ddc242-5e68-4cd0-9e0b-95ca9a78c0a5 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:50.173240 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ddc242-5e68-4cd0-9e0b-95ca9a78c0a5 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:50.173516 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ddc242-5e68-4cd0-9e0b-95ca9a78c0a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:50.186838 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1df9631-167d-47eb-8b0e-a1f6cd48b66b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:59:50.188308 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1df9631-167d-47eb-8b0e-a1f6cd48b66b None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:59:50.203680 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-82ddc242-5e68-4cd0-9e0b-95ca9a78c0a5 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:59:50.205072 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-82ddc242-5e68-4cd0-9e0b-95ca9a78c0a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:59:50.208483 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-e1df9631-167d-47eb-8b0e-a1f6cd48b66b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:59:50.208483 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:59:50.208483 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:59:50.208483 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:59:50.208483 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:59:50.208483 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:59:50.208483 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:59:50.208483 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:59:50.208483 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:59:50.208483 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:59:50.208483 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:59:50.208483 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:59:50.208483 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 02:59:50.208483 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 02:59:50.208483 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 02:59:50.208483 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 02:59:50.209540 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:59:50.209540 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:59:50.209540 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:59:50.209540 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:59:50.209540 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:59:50.209540 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:59:50.209540 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:59:50.209540 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 02:59:50.209540 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:59:50.209540 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:59:50.209540 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:59:50.209540 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:59:50.209540 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:59:50.209540 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:59:50.209540 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:59:50.209540 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:59:50.209540 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 02:59:50.210606 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 02:59:50.210606 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:59:50.210606 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 02:59:50.211969 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 14/31] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 02:59:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 02:59:50.214384 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1df9631-167d-47eb-8b0e-a1f6cd48b66b None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:59:50.279572 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:59:50.288648 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 18/32] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 02:59:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 02:59:50.290030 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ddc242-5e68-4cd0-9e0b-95ca9a78c0a5 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:59:50.353977 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1df9631-167d-47eb-8b0e-a1f6cd48b66b None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:59:50.411825 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ddc242-5e68-4cd0-9e0b-95ca9a78c0a5 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:59:50.439035 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a7152d-c6f0-4451-b62e-b7fa76eb0fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:59:50.439693 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a7152d-c6f0-4451-b62e-b7fa76eb0fa0 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:50.439977 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a7152d-c6f0-4451-b62e-b7fa76eb0fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:50.440255 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a7152d-c6f0-4451-b62e-b7fa76eb0fa0 None admin] PATH_INFO: `/v3/users/e8b708e7077c461bb73c230d7b1825fb` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:50.441884 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a7152d-c6f0-4451-b62e-b7fa76eb0fa0 None admin] RBAC: Authorizing `identity:get_user(user_id=e8b708e7077c461bb73c230d7b1825fb)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:59:50.443370 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a7152d-c6f0-4451-b62e-b7fa76eb0fa0 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:59:50.445141 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 15/33] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Aug 25 02:59:50 2022] GET /identity/v3/users/e8b708e7077c461bb73c230d7b1825fb => generated 247 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:59:50.447133 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a7152d-c6f0-4451-b62e-b7fa76eb0fa0 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:59:50.449261 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad135d8-9b99-4bb6-9546-8464ae27300b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:59:50.450376 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad135d8-9b99-4bb6-9546-8464ae27300b None admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:50.450834 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad135d8-9b99-4bb6-9546-8464ae27300b None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:50.451257 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad135d8-9b99-4bb6-9546-8464ae27300b None admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:50.452349 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad135d8-9b99-4bb6-9546-8464ae27300b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:59:50.453862 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad135d8-9b99-4bb6-9546-8464ae27300b None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:59:50.483304 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:59:50.561840 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a7152d-c6f0-4451-b62e-b7fa76eb0fa0 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:59:50.603273 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df23df1f-8982-4c58-8760-7f6b903af815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:59:50.603905 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df23df1f-8982-4c58-8760-7f6b903af815 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:50.604191 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df23df1f-8982-4c58-8760-7f6b903af815 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:50.604470 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df23df1f-8982-4c58-8760-7f6b903af815 None admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:50.605131 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df23df1f-8982-4c58-8760-7f6b903af815 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:59:50.605448 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df23df1f-8982-4c58-8760-7f6b903af815 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:59:50.611073 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df23df1f-8982-4c58-8760-7f6b903af815 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:59:50.630858 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 16/34] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Aug 25 02:59:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:59:50.632014 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df23df1f-8982-4c58-8760-7f6b903af815 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:59:50.734080 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:59:50.745767 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 19/35] 10.222.0.58 () {60 vars in 1131 bytes} [Thu Aug 25 02:59:50 2022] POST /identity/v3/roles => generated 219 bytes in 456 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 02:59:50.747109 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad135d8-9b99-4bb6-9546-8464ae27300b None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:59:50.760095 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df23df1f-8982-4c58-8760-7f6b903af815 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:59:50.812268 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521e50ed-06c3-46ad-a819-2aca6d22ea30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:59:50.812933 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521e50ed-06c3-46ad-a819-2aca6d22ea30 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:50.813218 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521e50ed-06c3-46ad-a819-2aca6d22ea30 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:50.813497 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521e50ed-06c3-46ad-a819-2aca6d22ea30 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:50.816097 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521e50ed-06c3-46ad-a819-2aca6d22ea30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:59:50.816261 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521e50ed-06c3-46ad-a819-2aca6d22ea30 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:59:50.818169 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 17/36] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 02:59:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:59:50.818985 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521e50ed-06c3-46ad-a819-2aca6d22ea30 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:59:50.870657 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:59:50.933087 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521e50ed-06c3-46ad-a819-2aca6d22ea30 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:59:50.946616 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad135d8-9b99-4bb6-9546-8464ae27300b None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:59:50.971820 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e13cdfc-3de6-46cb-8032-9bd6b7a9b597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:59:50.972524 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e13cdfc-3de6-46cb-8032-9bd6b7a9b597 None admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:50.972847 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e13cdfc-3de6-46cb-8032-9bd6b7a9b597 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:50.973134 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e13cdfc-3de6-46cb-8032-9bd6b7a9b597 None admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:50.973796 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e13cdfc-3de6-46cb-8032-9bd6b7a9b597 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:59:50.975614 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e13cdfc-3de6-46cb-8032-9bd6b7a9b597 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:59:50.986976 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d68b98-0b33-4a3b-b1cc-418d2ade044f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:59:50.987535 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d68b98-0b33-4a3b-b1cc-418d2ade044f None admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:50.987821 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d68b98-0b33-4a3b-b1cc-418d2ade044f None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:50.988097 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d68b98-0b33-4a3b-b1cc-418d2ade044f None admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:50.988779 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d68b98-0b33-4a3b-b1cc-418d2ade044f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:59:50.990420 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d68b98-0b33-4a3b-b1cc-418d2ade044f None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:59:51.091905 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 20/37] 10.222.0.58 () {60 vars in 1131 bytes} [Thu Aug 25 02:59:50 2022] POST /identity/v3/roles => generated 225 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 02:59:51.093582 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d68b98-0b33-4a3b-b1cc-418d2ade044f None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:59:51.186320 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 18/38] 10.222.0.58 () {60 vars in 1141 bytes} [Thu Aug 25 02:59:50 2022] POST /identity/v3/projects => generated 301 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 02:59:51.186320 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f42296-0612-4a5b-ba72-4341af472f87 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:51.186320 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f42296-0612-4a5b-ba72-4341af472f87 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:51.186320 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f42296-0612-4a5b-ba72-4341af472f87 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:51.186320 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 19/39] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 02:59:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:59:51.186320 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94b8406-d8a0-4175-a992-2f43a2edf82e None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:51.186320 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94b8406-d8a0-4175-a992-2f43a2edf82e None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:51.187409 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94b8406-d8a0-4175-a992-2f43a2edf82e None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:51.187409 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 20/40] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 02:59:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:59:51.190043 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a69d7a-29fe-46d6-b2a5-af4c5fb6d323 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:51.190043 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a69d7a-29fe-46d6-b2a5-af4c5fb6d323 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:51.190043 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a69d7a-29fe-46d6-b2a5-af4c5fb6d323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:51.212934 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:59:51.215283 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-f7a69d7a-29fe-46d6-b2a5-af4c5fb6d323 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:59:51.216925 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-f7a69d7a-29fe-46d6-b2a5-af4c5fb6d323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:59:51.251413 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d68b98-0b33-4a3b-b1cc-418d2ade044f None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:59:51.277407 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:59:51.306861 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68060cde-b2c1-4223-a398-2df158d5a818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:59:51.308370 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68060cde-b2c1-4223-a398-2df158d5a818 None admin] REQUEST_METHOD: `PUT` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:51.308370 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68060cde-b2c1-4223-a398-2df158d5a818 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:51.308370 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68060cde-b2c1-4223-a398-2df158d5a818 None admin] PATH_INFO: `/v3/domains/default/users/e8b708e7077c461bb73c230d7b1825fb/roles/62f19ea272b447ccaab1576cd9bf4fd9` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:51.311813 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68060cde-b2c1-4223-a398-2df158d5a818 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=e8b708e7077c461bb73c230d7b1825fb, role_id=62f19ea272b447ccaab1576cd9bf4fd9)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:59:51.313682 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68060cde-b2c1-4223-a398-2df158d5a818 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:59:51.325586 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 21/41] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 02:59:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 25 02:59:51.328490 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0588a97-e328-4174-bc20-3eb4e6af4f2a None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:51.329762 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0588a97-e328-4174-bc20-3eb4e6af4f2a None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:51.329762 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0588a97-e328-4174-bc20-3eb4e6af4f2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:51.351534 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-b0588a97-e328-4174-bc20-3eb4e6af4f2a None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:59:51.352737 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-b0588a97-e328-4174-bc20-3eb4e6af4f2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:59:51.383647 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 21/42] 10.222.0.58 () {58 vars in 1361 bytes} [Thu Aug 25 02:59:51 2022] PUT /identity/v3/domains/default/users/e8b708e7077c461bb73c230d7b1825fb/roles/62f19ea272b447ccaab1576cd9bf4fd9 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 02:59:51.445284 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 22/43] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 02:59:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 02:59:51.598669 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68060cde-b2c1-4223-a398-2df158d5a818 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:59:51.718152 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0588a97-e328-4174-bc20-3eb4e6af4f2a None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:59:51.788175 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:59:51.810661 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68060cde-b2c1-4223-a398-2df158d5a818 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:59:51.829845 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0588a97-e328-4174-bc20-3eb4e6af4f2a None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:59:51.853338 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f32bd67-aa5d-4c1f-9094-1fad1e2a6ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:59:51.853890 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f32bd67-aa5d-4c1f-9094-1fad1e2a6ccc None admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:51.854171 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f32bd67-aa5d-4c1f-9094-1fad1e2a6ccc None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:51.854487 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f32bd67-aa5d-4c1f-9094-1fad1e2a6ccc None admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:51.855164 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f32bd67-aa5d-4c1f-9094-1fad1e2a6ccc None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:59:51.856672 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f32bd67-aa5d-4c1f-9094-1fad1e2a6ccc None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:59:51.865926 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0cff4a-22be-4cae-aa85-f858abc79a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:59:51.866682 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0cff4a-22be-4cae-aa85-f858abc79a88 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:51.867027 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0cff4a-22be-4cae-aa85-f858abc79a88 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:51.867365 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0cff4a-22be-4cae-aa85-f858abc79a88 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:51.868631 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0cff4a-22be-4cae-aa85-f858abc79a88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:59:51.870313 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0cff4a-22be-4cae-aa85-f858abc79a88 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:59:51.872088 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 23/44] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 02:59:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:59:51.877899 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0cff4a-22be-4cae-aa85-f858abc79a88 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:59:51.928710 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 22/45] 10.222.0.58 () {60 vars in 1131 bytes} [Thu Aug 25 02:59:51 2022] POST /identity/v3/roles => generated 223 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 02:59:51.987525 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0cff4a-22be-4cae-aa85-f858abc79a88 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:59:52.026783 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3871c5-cfab-4f66-9582-4184727cef93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:59:52.027659 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3871c5-cfab-4f66-9582-4184727cef93 None admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:52.028089 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3871c5-cfab-4f66-9582-4184727cef93 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:52.028619 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3871c5-cfab-4f66-9582-4184727cef93 None admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:52.029478 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3871c5-cfab-4f66-9582-4184727cef93 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:59:52.031798 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3871c5-cfab-4f66-9582-4184727cef93 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:59:52.037074 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:59:52.172865 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 24/46] 10.222.0.58 () {60 vars in 1141 bytes} [Thu Aug 25 02:59:51 2022] POST /identity/v3/projects => generated 312 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 02:59:53.320081 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dc955f-2760-48fc-8ea1-88a484077062 None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:53.320596 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dc955f-2760-48fc-8ea1-88a484077062 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:53.320678 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dc955f-2760-48fc-8ea1-88a484077062 None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:53.321980 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 23/47] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 02:59:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:59:53.327731 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d544f4-8e1e-4e88-9450-17518af2b397 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:53.328150 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d544f4-8e1e-4e88-9450-17518af2b397 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:53.328407 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d544f4-8e1e-4e88-9450-17518af2b397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:53.449818 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-a2d544f4-8e1e-4e88-9450-17518af2b397 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:59:53.451052 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-a2d544f4-8e1e-4e88-9450-17518af2b397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:59:53.520527 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 25/48] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 02:59:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 02:59:53.788982 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dc955f-2760-48fc-8ea1-88a484077062 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:59:53.824692 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:59:53.889620 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dc955f-2760-48fc-8ea1-88a484077062 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:59:53.922963 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7c6b64-ef8b-4829-8c6f-53c595201447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:59:53.923597 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7c6b64-ef8b-4829-8c6f-53c595201447 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:53.923882 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7c6b64-ef8b-4829-8c6f-53c595201447 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:53.924159 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7c6b64-ef8b-4829-8c6f-53c595201447 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:53.933944 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7c6b64-ef8b-4829-8c6f-53c595201447 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:59:53.938682 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-7b7c6b64-ef8b-4829-8c6f-53c595201447 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 02:59:53.938682 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:59:53.938682 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:59:53.938682 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:59:53.938682 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:59:53.938682 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:59:53.938682 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:59:53.938682 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:59:53.938682 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:59:53.938682 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:59:53.938682 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:59:53.938682 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:59:53.938682 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:59:53.938682 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:59:53.938682 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:59:53.938682 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:59:53.939481 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:59:53.939481 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:59:53.939481 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:59:53.939481 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:59:53.939481 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:59:53.939481 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:59:53.939481 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:59:53.939481 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:59:53.939481 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:59:53.939481 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:59:53.939481 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:59:53.939481 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 02:59:53.939481 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:59:53.939481 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:59:53.939481 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:59:53.939481 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:59:53.940239 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:59:53.940239 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:59:53.940239 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:59:53.940239 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:59:53.940239 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:59:53.940239 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:59:53.940239 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:59:53.940239 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:59:53.940239 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:59:53.940239 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:59:53.940239 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 02:59:53.940239 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 02:59:53.940239 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 24/49] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Aug 25 02:59:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:59:53.945677 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d544f4-8e1e-4e88-9450-17518af2b397 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:59:54.042574 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d544f4-8e1e-4e88-9450-17518af2b397 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:59:54.058166 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:59:54.061273 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87261fda-85c8-485c-b86a-e44f4a11b8ad None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:54.061562 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87261fda-85c8-485c-b86a-e44f4a11b8ad None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:54.061849 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87261fda-85c8-485c-b86a-e44f4a11b8ad None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:54.063258 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 25/50] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 02:59:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 25 02:59:54.068214 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f4ea53-dfa7-43b3-83b5-f974cfca9ee7 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:54.068512 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f4ea53-dfa7-43b3-83b5-f974cfca9ee7 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:54.068790 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f4ea53-dfa7-43b3-83b5-f974cfca9ee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:54.078395 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146a8c07-3ffb-4581-ba48-b7da7018e126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:59:54.079022 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146a8c07-3ffb-4581-ba48-b7da7018e126 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:54.079306 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146a8c07-3ffb-4581-ba48-b7da7018e126 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:54.079582 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146a8c07-3ffb-4581-ba48-b7da7018e126 None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:54.080252 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146a8c07-3ffb-4581-ba48-b7da7018e126 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:59:54.080559 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146a8c07-3ffb-4581-ba48-b7da7018e126 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:59:54.081947 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146a8c07-3ffb-4581-ba48-b7da7018e126 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:59:54.086911 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-f5f4ea53-dfa7-43b3-83b5-f974cfca9ee7 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:59:54.088435 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-f5f4ea53-dfa7-43b3-83b5-f974cfca9ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:59:54.095467 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 26/51] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Aug 25 02:59:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:59:54.101084 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146a8c07-3ffb-4581-ba48-b7da7018e126 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:59:54.157121 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 26/52] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 02:59:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 25 02:59:54.199480 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146a8c07-3ffb-4581-ba48-b7da7018e126 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:59:54.233992 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e4786d-37c3-404d-b5e5-5a940026d78f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:59:54.234671 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e4786d-37c3-404d-b5e5-5a940026d78f None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:54.234960 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e4786d-37c3-404d-b5e5-5a940026d78f None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:54.235239 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e4786d-37c3-404d-b5e5-5a940026d78f None admin] PATH_INFO: `/v3/users/e8b708e7077c461bb73c230d7b1825fb` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:54.236378 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e4786d-37c3-404d-b5e5-5a940026d78f None admin] RBAC: Authorizing `identity:get_user(user_id=e8b708e7077c461bb73c230d7b1825fb)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:59:54.237834 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e4786d-37c3-404d-b5e5-5a940026d78f None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:59:54.239627 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 27/53] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Aug 25 02:59:54 2022] GET /identity/v3/users/e8b708e7077c461bb73c230d7b1825fb => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:59:54.244971 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f4ea53-dfa7-43b3-83b5-f974cfca9ee7 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:59:54.272934 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:59:54.344159 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f4ea53-dfa7-43b3-83b5-f974cfca9ee7 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:59:54.376713 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2ea192-e66c-4727-baa9-b09ab04165fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:59:54.377348 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2ea192-e66c-4727-baa9-b09ab04165fd None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:54.377629 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2ea192-e66c-4727-baa9-b09ab04165fd None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:54.377928 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2ea192-e66c-4727-baa9-b09ab04165fd None admin] PATH_INFO: `/v3/domains/default` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:54.379166 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2ea192-e66c-4727-baa9-b09ab04165fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:59:54.380623 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2ea192-e66c-4727-baa9-b09ab04165fd None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:59:54.382192 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 27/54] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 02:59:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:59:54.387910 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e4786d-37c3-404d-b5e5-5a940026d78f None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:59:54.422369 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2ea192-e66c-4727-baa9-b09ab04165fd None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:59:54.436707 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:59:54.489680 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e4786d-37c3-404d-b5e5-5a940026d78f None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:59:54.513598 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:59:54.520494 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2ea192-e66c-4727-baa9-b09ab04165fd None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:59:54.523478 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4283f063-fd66-4c1d-919b-869062086867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:59:54.523478 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4283f063-fd66-4c1d-919b-869062086867 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:54.523478 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4283f063-fd66-4c1d-919b-869062086867 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:54.523732 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4283f063-fd66-4c1d-919b-869062086867 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:54.524439 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4283f063-fd66-4c1d-919b-869062086867 None admin] RBAC: Adding query filter params (role.id=62f19ea272b447ccaab1576cd9bf4fd9, scope.domain.id=default, user.id=e8b708e7077c461bb73c230d7b1825fb) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:59:54.524751 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4283f063-fd66-4c1d-919b-869062086867 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:59:54.526186 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4283f063-fd66-4c1d-919b-869062086867 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:59:54.535194 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 28/55] 10.222.0.58 () {56 vars in 1322 bytes} [Thu Aug 25 02:59:54 2022] GET /identity/v3/role_assignments?user.id=e8b708e7077c461bb73c230d7b1825fb&scope.domain.id=default&role.id=62f19ea272b447ccaab1576cd9bf4fd9 => generated 530 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:59:54.554825 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d0db13-ea31-429d-93a4-764f4e009881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:59:54.555490 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d0db13-ea31-429d-93a4-764f4e009881 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:54.555775 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d0db13-ea31-429d-93a4-764f4e009881 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:54.556057 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d0db13-ea31-429d-93a4-764f4e009881 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:54.557259 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d0db13-ea31-429d-93a4-764f4e009881 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:59:54.558806 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d0db13-ea31-429d-93a4-764f4e009881 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:59:54.560658 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 28/56] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 02:59:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:59:54.566958 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4283f063-fd66-4c1d-919b-869062086867 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:59:54.676298 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4283f063-fd66-4c1d-919b-869062086867 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:59:54.688640 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:59:54.712558 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f086cd91-4e2d-440a-94bc-32f4b2229348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:59:54.713187 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f086cd91-4e2d-440a-94bc-32f4b2229348 None admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:54.713471 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f086cd91-4e2d-440a-94bc-32f4b2229348 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:54.713751 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f086cd91-4e2d-440a-94bc-32f4b2229348 None admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:54.714502 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f086cd91-4e2d-440a-94bc-32f4b2229348 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:59:54.715952 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f086cd91-4e2d-440a-94bc-32f4b2229348 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:59:54.801342 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 29/57] 10.222.0.58 () {60 vars in 1140 bytes} [Thu Aug 25 02:59:54 2022] POST /identity/v3/projects => generated 298 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 02:59:56.689649 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9605a1e-37f1-4572-bf10-8c06d2f9493e None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:56.690111 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9605a1e-37f1-4572-bf10-8c06d2f9493e None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:56.690200 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9605a1e-37f1-4572-bf10-8c06d2f9493e None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:56.691575 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 29/58] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 02:59:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:59:56.696867 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1114ad6-03bf-485e-8728-3bb31c60684a None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:56.697231 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1114ad6-03bf-485e-8728-3bb31c60684a None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:56.697434 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1114ad6-03bf-485e-8728-3bb31c60684a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:56.715643 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-d1114ad6-03bf-485e-8728-3bb31c60684a None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:59:56.716821 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-d1114ad6-03bf-485e-8728-3bb31c60684a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:59:56.783873 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 30/59] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 02:59:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 02:59:57.050227 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9605a1e-37f1-4572-bf10-8c06d2f9493e None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:59:57.148533 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9605a1e-37f1-4572-bf10-8c06d2f9493e None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:59:57.182346 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1febd3c-32c3-41bb-bef0-df83b8d388f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:59:57.182983 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1febd3c-32c3-41bb-bef0-df83b8d388f7 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:57.183270 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1febd3c-32c3-41bb-bef0-df83b8d388f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:57.183552 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1febd3c-32c3-41bb-bef0-df83b8d388f7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:57.184672 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1febd3c-32c3-41bb-bef0-df83b8d388f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:59:57.186845 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1febd3c-32c3-41bb-bef0-df83b8d388f7 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:59:57.188417 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 30/60] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 02:59:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:59:57.194060 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1114ad6-03bf-485e-8728-3bb31c60684a None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:59:57.282246 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:59:57.290598 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1114ad6-03bf-485e-8728-3bb31c60684a None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:59:57.323264 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20732580-77e5-417a-9a98-f6e247fc3876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:59:57.323862 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20732580-77e5-417a-9a98-f6e247fc3876 None admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:57.324153 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20732580-77e5-417a-9a98-f6e247fc3876 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:57.324433 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20732580-77e5-417a-9a98-f6e247fc3876 None admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:57.325078 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20732580-77e5-417a-9a98-f6e247fc3876 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:59:57.326566 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20732580-77e5-417a-9a98-f6e247fc3876 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:59:57.525175 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 31/61] 10.222.0.58 () {60 vars in 1132 bytes} [Thu Aug 25 02:59:57 2022] POST /identity/v3/users => generated 275 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:00:00.498472 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ecd5d8-4321-42ea-837e-a5ce2b39c4ba None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:00.500022 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ecd5d8-4321-42ea-837e-a5ce2b39c4ba None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:00.500537 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ecd5d8-4321-42ea-837e-a5ce2b39c4ba None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:00.502147 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 31/62] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:00:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:00:00.507899 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d060e6c6-5c53-4f6c-8907-29cacbd576ff None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:00.508752 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d060e6c6-5c53-4f6c-8907-29cacbd576ff None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:00.509238 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d060e6c6-5c53-4f6c-8907-29cacbd576ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:00.528321 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-d060e6c6-5c53-4f6c-8907-29cacbd576ff None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:00:00.529835 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-d060e6c6-5c53-4f6c-8907-29cacbd576ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:00.589533 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996ae20d-8e52-4fee-ae0a-6cc0bebb9560 None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:00.590133 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996ae20d-8e52-4fee-ae0a-6cc0bebb9560 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:00.590628 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996ae20d-8e52-4fee-ae0a-6cc0bebb9560 None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:00.592187 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 32/63] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:00:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:00:00.597028 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906795e5-6490-4b88-9f40-b4204f0bd7c2 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:00.597547 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906795e5-6490-4b88-9f40-b4204f0bd7c2 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:00.598019 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906795e5-6490-4b88-9f40-b4204f0bd7c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:00.618470 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-906795e5-6490-4b88-9f40-b4204f0bd7c2 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:00:00.620577 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-906795e5-6490-4b88-9f40-b4204f0bd7c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:00.627162 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 32/64] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:00:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 03:00:00.660569 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:00.718921 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 33/65] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:00:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 25 03:00:00.965333 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d060e6c6-5c53-4f6c-8907-29cacbd576ff None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:01.004102 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a247a08f-b7a3-44c3-9b72-900740ed024a None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:01.004689 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a247a08f-b7a3-44c3-9b72-900740ed024a None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:01.005125 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a247a08f-b7a3-44c3-9b72-900740ed024a None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:01.006669 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 34/66] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:00:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:00:01.011627 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ec7d65-dea4-40f2-83b4-17ce809ec2c0 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:01.012152 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ec7d65-dea4-40f2-83b4-17ce809ec2c0 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:01.012646 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ec7d65-dea4-40f2-83b4-17ce809ec2c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:01.041648 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-08ec7d65-dea4-40f2-83b4-17ce809ec2c0 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:00:01.043188 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-08ec7d65-dea4-40f2-83b4-17ce809ec2c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:01.065161 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:01.101985 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d060e6c6-5c53-4f6c-8907-29cacbd576ff None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:01.140587 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 35/67] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:00:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 25 03:00:01.142905 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d397e5e-93b1-44d7-a173-7c621154e7b9 None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:01.143416 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d397e5e-93b1-44d7-a173-7c621154e7b9 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:01.143887 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d397e5e-93b1-44d7-a173-7c621154e7b9 None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:01.145355 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 36/68] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:00:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:00:01.156844 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9f64b9-2bd2-48d0-8723-b694e960b08e None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:01.157399 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9f64b9-2bd2-48d0-8723-b694e960b08e None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:01.157903 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9f64b9-2bd2-48d0-8723-b694e960b08e None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:01.159345 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 37/69] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:00:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:00:01.160500 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aa975e-495e-4f68-a384-29f111aa7e44 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:01.160811 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aa975e-495e-4f68-a384-29f111aa7e44 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:01.161286 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aa975e-495e-4f68-a384-29f111aa7e44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:01.165336 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df69ebbf-8d7a-49a6-b5c1-9d306f27d688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:01.166000 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df69ebbf-8d7a-49a6-b5c1-9d306f27d688 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:01.166379 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df69ebbf-8d7a-49a6-b5c1-9d306f27d688 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:01.166597 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df69ebbf-8d7a-49a6-b5c1-9d306f27d688 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:01.178553 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df69ebbf-8d7a-49a6-b5c1-9d306f27d688 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:00:01.179778 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-f3aa975e-495e-4f68-a384-29f111aa7e44 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:00:01.180981 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-f3aa975e-495e-4f68-a384-29f111aa7e44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:01.184840 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-df69ebbf-8d7a-49a6-b5c1-9d306f27d688 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 25 03:00:01.184840 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:00:01.184840 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:00:01.184840 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:00:01.184840 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:00:01.184840 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:00:01.184840 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:00:01.184840 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:00:01.184840 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:00:01.184840 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:00:01.184840 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:00:01.184840 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:00:01.184840 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:00:01.184840 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:00:01.184840 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:00:01.184840 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:00:01.185688 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:00:01.185688 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:00:01.185688 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:00:01.185688 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:00:01.185688 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:00:01.185688 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:00:01.185688 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:00:01.185688 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:00:01.185688 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:00:01.185688 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:00:01.185688 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:00:01.185688 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:00:01.185688 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:00:01.185688 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:00:01.185688 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:00:01.185688 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:00:01.187622 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:00:01.187622 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:00:01.187622 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:00:01.187622 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:00:01.187622 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:00:01.187622 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:00:01.187622 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:00:01.187622 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:00:01.187622 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:00:01.187622 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:00:01.187622 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 25 03:00:01.187622 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:00:01.187622 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 33/70] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Aug 25 03:00:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:00:01.187622 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df69ebbf-8d7a-49a6-b5c1-9d306f27d688 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:01.254552 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 38/71] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:00:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 03:00:01.259561 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99b45d7-f069-446f-ab03-2280529ea191 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:01.259853 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99b45d7-f069-446f-ab03-2280529ea191 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:01.260138 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99b45d7-f069-446f-ab03-2280529ea191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:01.278596 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-c99b45d7-f069-446f-ab03-2280529ea191 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:00:01.279866 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-c99b45d7-f069-446f-ab03-2280529ea191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:01.295211 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:01.313303 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df69ebbf-8d7a-49a6-b5c1-9d306f27d688 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:01.349622 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 39/72] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:00:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 03:00:01.356227 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2dee0d-c4f2-4104-a6d7-3b93172f3634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:01.356824 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99b45d7-f069-446f-ab03-2280529ea191 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:01.357338 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2dee0d-c4f2-4104-a6d7-3b93172f3634 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:01.357798 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2dee0d-c4f2-4104-a6d7-3b93172f3634 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:01.358790 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2dee0d-c4f2-4104-a6d7-3b93172f3634 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:01.366994 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:01.371962 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2dee0d-c4f2-4104-a6d7-3b93172f3634 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:00:01.378966 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-ef2dee0d-c4f2-4104-a6d7-3b93172f3634 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 03:00:01.378966 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:00:01.378966 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:00:01.378966 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:00:01.378966 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:00:01.378966 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:00:01.378966 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:00:01.378966 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:00:01.378966 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:00:01.378966 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:00:01.378966 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:00:01.378966 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:00:01.378966 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:00:01.378966 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:00:01.378966 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:00:01.378966 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:00:01.386194 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:00:01.386194 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:00:01.386194 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:00:01.386194 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:00:01.386194 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:00:01.386194 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:00:01.386194 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:00:01.386194 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:00:01.386194 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:00:01.386194 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:00:01.386194 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:00:01.386194 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:00:01.386194 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:00:01.386194 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:00:01.386194 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:00:01.386194 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:00:01.386981 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:00:01.386981 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:00:01.386981 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:00:01.386981 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:00:01.386981 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:00:01.386981 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:00:01.386981 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:00:01.386981 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:00:01.386981 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:00:01.386981 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:00:01.386981 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 03:00:01.386981 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:00:01.386981 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 34/73] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Aug 25 03:00:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:00:01.388830 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2dee0d-c4f2-4104-a6d7-3b93172f3634 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:01.495925 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2dee0d-c4f2-4104-a6d7-3b93172f3634 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:01.517905 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99b45d7-f069-446f-ab03-2280529ea191 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:01.531777 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a335f9fd-9e13-4a5c-a3a0-b7686e793492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:01.532404 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a335f9fd-9e13-4a5c-a3a0-b7686e793492 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:01.532690 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a335f9fd-9e13-4a5c-a3a0-b7686e793492 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:01.532965 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a335f9fd-9e13-4a5c-a3a0-b7686e793492 None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:01.533671 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a335f9fd-9e13-4a5c-a3a0-b7686e793492 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:01.533940 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a335f9fd-9e13-4a5c-a3a0-b7686e793492 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:01.535385 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a335f9fd-9e13-4a5c-a3a0-b7686e793492 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:01.543634 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 35/74] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Aug 25 03:00:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:01.544422 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a335f9fd-9e13-4a5c-a3a0-b7686e793492 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:01.558587 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026fcfea-aaea-48ec-8c08-48f696c6413f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:01.559204 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026fcfea-aaea-48ec-8c08-48f696c6413f None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:01.559493 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026fcfea-aaea-48ec-8c08-48f696c6413f None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:01.559771 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026fcfea-aaea-48ec-8c08-48f696c6413f None admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:01.560439 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026fcfea-aaea-48ec-8c08-48f696c6413f None admin] RBAC: Adding query filter params (name=member) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:01.560747 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026fcfea-aaea-48ec-8c08-48f696c6413f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:01.562870 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026fcfea-aaea-48ec-8c08-48f696c6413f None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:01.574425 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 40/75] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Aug 25 03:00:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:01.574425 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026fcfea-aaea-48ec-8c08-48f696c6413f None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:01.588577 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:01.618326 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:01.664937 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a335f9fd-9e13-4a5c-a3a0-b7686e793492 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:01.674721 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026fcfea-aaea-48ec-8c08-48f696c6413f None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:01.698872 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6337e9a7-25a5-491e-9322-1a3a89805c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:01.699507 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6337e9a7-25a5-491e-9322-1a3a89805c2e None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:01.699794 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6337e9a7-25a5-491e-9322-1a3a89805c2e None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:01.700070 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6337e9a7-25a5-491e-9322-1a3a89805c2e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:01.708110 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88cd072-e6aa-4241-aa33-c937ff6d44a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:01.708737 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88cd072-e6aa-4241-aa33-c937ff6d44a5 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:01.709018 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88cd072-e6aa-4241-aa33-c937ff6d44a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:01.709291 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88cd072-e6aa-4241-aa33-c937ff6d44a5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:01.710253 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6337e9a7-25a5-491e-9322-1a3a89805c2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:00:01.710429 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88cd072-e6aa-4241-aa33-c937ff6d44a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:01.711872 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88cd072-e6aa-4241-aa33-c937ff6d44a5 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:01.713433 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 41/76] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:00:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:01.714185 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88cd072-e6aa-4241-aa33-c937ff6d44a5 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:01.715057 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-6337e9a7-25a5-491e-9322-1a3a89805c2e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 25 03:00:01.715057 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:00:01.715057 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:00:01.715057 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:00:01.715057 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:00:01.715057 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:00:01.715057 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:00:01.715057 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:00:01.715057 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:00:01.715057 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:00:01.715057 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:00:01.715057 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:00:01.715057 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:00:01.715057 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:00:01.715057 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:00:01.715057 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:00:01.715874 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:00:01.715874 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:00:01.715874 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:00:01.715874 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:00:01.715874 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:00:01.715874 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:00:01.715874 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:00:01.715874 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:00:01.715874 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:00:01.715874 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:00:01.715874 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:00:01.715874 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:00:01.715874 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:00:01.715874 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:00:01.715874 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:00:01.715874 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:00:01.716705 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:00:01.716705 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:00:01.716705 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:00:01.716705 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:00:01.716705 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:00:01.716705 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:00:01.716705 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:00:01.716705 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:00:01.716705 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:00:01.716705 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:00:01.716705 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 25 03:00:01.716705 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:00:01.716705 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 36/77] 10.222.0.58 () {56 vars in 1114 bytes} [Thu Aug 25 03:00:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:00:01.718093 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6337e9a7-25a5-491e-9322-1a3a89805c2e None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:01.801779 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:01.821117 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88cd072-e6aa-4241-aa33-c937ff6d44a5 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:01.824258 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6337e9a7-25a5-491e-9322-1a3a89805c2e None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:01.844995 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:01.855144 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0a890b-95b4-4772-94d8-8e3282205b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:01.855778 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0a890b-95b4-4772-94d8-8e3282205b96 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:01.856059 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0a890b-95b4-4772-94d8-8e3282205b96 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:01.856334 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0a890b-95b4-4772-94d8-8e3282205b96 None admin] PATH_INFO: `/v3/users/e8b708e7077c461bb73c230d7b1825fb` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:01.857522 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0a890b-95b4-4772-94d8-8e3282205b96 None admin] RBAC: Authorizing `identity:get_user(user_id=e8b708e7077c461bb73c230d7b1825fb)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:01.859029 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17228ef3-5244-4897-9acb-785345cc2c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:01.859566 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0a890b-95b4-4772-94d8-8e3282205b96 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:01.859655 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17228ef3-5244-4897-9acb-785345cc2c2e None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:01.859934 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17228ef3-5244-4897-9acb-785345cc2c2e None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:01.860618 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17228ef3-5244-4897-9acb-785345cc2c2e None admin] PATH_INFO: `/v3/users/26e7c9fc5d3c41de8156071d9f0cb3cd` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:01.861209 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 42/78] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Aug 25 03:00:01 2022] GET /identity/v3/users/e8b708e7077c461bb73c230d7b1825fb => generated 247 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:01.863116 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0a890b-95b4-4772-94d8-8e3282205b96 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:01.875430 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17228ef3-5244-4897-9acb-785345cc2c2e None admin] RBAC: Authorizing `identity:get_user(user_id=26e7c9fc5d3c41de8156071d9f0cb3cd)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:01.876939 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17228ef3-5244-4897-9acb-785345cc2c2e None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:01.878887 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 37/79] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Aug 25 03:00:01 2022] GET /identity/v3/users/26e7c9fc5d3c41de8156071d9f0cb3cd => generated 275 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:01.879986 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17228ef3-5244-4897-9acb-785345cc2c2e None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:01.961950 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0a890b-95b4-4772-94d8-8e3282205b96 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:01.979402 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17228ef3-5244-4897-9acb-785345cc2c2e None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:01.996244 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73509159-df34-4a2a-9555-a6281836dab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:01.996871 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73509159-df34-4a2a-9555-a6281836dab3 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:01.997151 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73509159-df34-4a2a-9555-a6281836dab3 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:01.997423 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73509159-df34-4a2a-9555-a6281836dab3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:02.007197 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73509159-df34-4a2a-9555-a6281836dab3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:00:02.011836 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-73509159-df34-4a2a-9555-a6281836dab3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 25 03:00:02.011836 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:00:02.011836 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:00:02.011836 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:00:02.011836 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:00:02.011836 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:00:02.011836 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:00:02.011836 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:00:02.011836 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:00:02.011836 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:00:02.011836 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:00:02.011836 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:00:02.011836 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:00:02.011836 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:00:02.011836 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:00:02.011836 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:00:02.012740 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:00:02.012740 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:00:02.012740 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:00:02.012740 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:00:02.012740 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:00:02.012740 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:00:02.012740 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:00:02.012740 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:00:02.012740 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:00:02.012740 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:00:02.012740 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:00:02.012740 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:00:02.012740 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:00:02.012740 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:00:02.012740 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:00:02.012740 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:00:02.013484 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:00:02.013484 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:00:02.013484 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:00:02.013484 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:00:02.013484 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:00:02.013484 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:00:02.013484 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:00:02.013484 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:00:02.013484 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:00:02.013484 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:00:02.013484 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 25 03:00:02.013484 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:00:02.014041 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da649ed-80e8-4a9f-8b1e-c56bca29ece5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:02.014121 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 43/80] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Aug 25 03:00:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:00:02.014185 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da649ed-80e8-4a9f-8b1e-c56bca29ece5 None admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:02.014185 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da649ed-80e8-4a9f-8b1e-c56bca29ece5 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:02.014335 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73509159-df34-4a2a-9555-a6281836dab3 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:02.014402 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da649ed-80e8-4a9f-8b1e-c56bca29ece5 None admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:02.014985 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da649ed-80e8-4a9f-8b1e-c56bca29ece5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:02.016185 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da649ed-80e8-4a9f-8b1e-c56bca29ece5 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:02.024748 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:02.115059 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73509159-df34-4a2a-9555-a6281836dab3 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:02.133416 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 38/81] 10.222.0.58 () {60 vars in 1132 bytes} [Thu Aug 25 03:00:01 2022] POST /identity/v3/users => generated 289 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:00:02.134443 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da649ed-80e8-4a9f-8b1e-c56bca29ece5 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:02.157241 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedc0038-2d43-4e31-b000-4c04665b5fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:02.157882 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedc0038-2d43-4e31-b000-4c04665b5fc3 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:02.158546 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedc0038-2d43-4e31-b000-4c04665b5fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:02.158546 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedc0038-2d43-4e31-b000-4c04665b5fc3 None admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:02.159316 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedc0038-2d43-4e31-b000-4c04665b5fc3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:02.159626 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedc0038-2d43-4e31-b000-4c04665b5fc3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:02.161199 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedc0038-2d43-4e31-b000-4c04665b5fc3 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:02.170894 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 44/82] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Aug 25 03:00:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:02.171779 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedc0038-2d43-4e31-b000-4c04665b5fc3 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:02.180259 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:02.251456 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:02.256332 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da649ed-80e8-4a9f-8b1e-c56bca29ece5 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:02.285892 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedc0038-2d43-4e31-b000-4c04665b5fc3 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:02.301642 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbc3800-92c6-41b4-8424-e88f74b0e53c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:02.302630 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbc3800-92c6-41b4-8424-e88f74b0e53c None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:02.302919 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbc3800-92c6-41b4-8424-e88f74b0e53c None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:02.303211 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbc3800-92c6-41b4-8424-e88f74b0e53c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:02.324954 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c61dae3-3456-4604-b1db-58fe90daa71c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:02.325573 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c61dae3-3456-4604-b1db-58fe90daa71c None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:02.325861 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c61dae3-3456-4604-b1db-58fe90daa71c None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:02.326142 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c61dae3-3456-4604-b1db-58fe90daa71c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:02.386380 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbc3800-92c6-41b4-8424-e88f74b0e53c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:02.387926 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbc3800-92c6-41b4-8424-e88f74b0e53c None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:02.411721 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c61dae3-3456-4604-b1db-58fe90daa71c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:02.413214 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c61dae3-3456-4604-b1db-58fe90daa71c None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:02.430889 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:02.519284 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:02.521367 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 39/83] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Aug 25 03:00:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 25 03:00:02.522171 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbc3800-92c6-41b4-8424-e88f74b0e53c None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:02.553858 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 45/84] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Aug 25 03:00:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 25 03:00:02.555106 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c61dae3-3456-4604-b1db-58fe90daa71c None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:02.625446 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbc3800-92c6-41b4-8424-e88f74b0e53c None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:02.660936 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2197de8e-69f7-42e2-97ef-f56d337e7988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:02.661738 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2197de8e-69f7-42e2-97ef-f56d337e7988 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:02.662062 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2197de8e-69f7-42e2-97ef-f56d337e7988 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:02.662468 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2197de8e-69f7-42e2-97ef-f56d337e7988 None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:02.663268 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2197de8e-69f7-42e2-97ef-f56d337e7988 None admin] RBAC: Adding query filter params (name=member) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:02.663692 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2197de8e-69f7-42e2-97ef-f56d337e7988 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:02.665171 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2197de8e-69f7-42e2-97ef-f56d337e7988 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:02.666955 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:02.672151 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c61dae3-3456-4604-b1db-58fe90daa71c None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:02.675014 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 40/85] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Aug 25 03:00:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:02.676472 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2197de8e-69f7-42e2-97ef-f56d337e7988 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:02.713542 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b9176a-1bf3-4d99-adeb-089af40607d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:02.714457 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b9176a-1bf3-4d99-adeb-089af40607d1 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:02.714880 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b9176a-1bf3-4d99-adeb-089af40607d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:02.715296 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b9176a-1bf3-4d99-adeb-089af40607d1 None admin] PATH_INFO: `/v3/users/26e7c9fc5d3c41de8156071d9f0cb3cd` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:02.716684 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b9176a-1bf3-4d99-adeb-089af40607d1 None admin] RBAC: Authorizing `identity:get_user(user_id=26e7c9fc5d3c41de8156071d9f0cb3cd)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:02.718499 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b9176a-1bf3-4d99-adeb-089af40607d1 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:02.720236 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 46/86] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Aug 25 03:00:02 2022] GET /identity/v3/users/26e7c9fc5d3c41de8156071d9f0cb3cd => generated 275 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:02.721210 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b9176a-1bf3-4d99-adeb-089af40607d1 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:02.765082 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:02.785634 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2197de8e-69f7-42e2-97ef-f56d337e7988 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:02.828946 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c959caf-ea17-474a-9538-2f8598b2e192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:02.829914 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c959caf-ea17-474a-9538-2f8598b2e192 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:02.830314 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c959caf-ea17-474a-9538-2f8598b2e192 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:02.830855 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c959caf-ea17-474a-9538-2f8598b2e192 None admin] PATH_INFO: `/v3/projects/0d32389226ec4d2dac8c390a33d5436c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:02.832211 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c959caf-ea17-474a-9538-2f8598b2e192 None admin] RBAC: Authorizing `identity:get_project(project_id=0d32389226ec4d2dac8c390a33d5436c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:02.833673 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c959caf-ea17-474a-9538-2f8598b2e192 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:02.835392 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 41/87] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Aug 25 03:00:02 2022] GET /identity/v3/projects/0d32389226ec4d2dac8c390a33d5436c => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:02.836250 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c959caf-ea17-474a-9538-2f8598b2e192 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:02.841653 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b9176a-1bf3-4d99-adeb-089af40607d1 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:02.889327 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c295d768-9495-401d-ac54-64e724a88b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:02.889991 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c295d768-9495-401d-ac54-64e724a88b3e None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:02.890261 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c295d768-9495-401d-ac54-64e724a88b3e None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:02.890569 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c295d768-9495-401d-ac54-64e724a88b3e None admin] PATH_INFO: `/v3/projects/0d32389226ec4d2dac8c390a33d5436c` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:02.891857 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c295d768-9495-401d-ac54-64e724a88b3e None admin] RBAC: Authorizing `identity:get_project(project_id=0d32389226ec4d2dac8c390a33d5436c)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:02.893296 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c295d768-9495-401d-ac54-64e724a88b3e None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:02.895159 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 47/88] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Aug 25 03:00:02 2022] GET /identity/v3/projects/0d32389226ec4d2dac8c390a33d5436c => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:02.895845 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c295d768-9495-401d-ac54-64e724a88b3e None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:02.913762 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:02.964554 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c959caf-ea17-474a-9538-2f8598b2e192 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:03.005536 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c295d768-9495-401d-ac54-64e724a88b3e None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:03.007657 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac34a0a-129a-46a1-b704-5afc1de2bafd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:03.008944 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac34a0a-129a-46a1-b704-5afc1de2bafd None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:03.009421 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac34a0a-129a-46a1-b704-5afc1de2bafd None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:03.009884 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac34a0a-129a-46a1-b704-5afc1de2bafd None admin] PATH_INFO: `/v3/users/26e7c9fc5d3c41de8156071d9f0cb3cd` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:03.011251 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac34a0a-129a-46a1-b704-5afc1de2bafd None admin] RBAC: Authorizing `identity:get_user(user_id=26e7c9fc5d3c41de8156071d9f0cb3cd)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:03.012897 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac34a0a-129a-46a1-b704-5afc1de2bafd None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:03.014749 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 42/89] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Aug 25 03:00:02 2022] GET /identity/v3/users/26e7c9fc5d3c41de8156071d9f0cb3cd => generated 275 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:03.015705 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac34a0a-129a-46a1-b704-5afc1de2bafd None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:03.037937 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:03.068999 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d63039f-97ef-40e7-ada7-023397173098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:03.069915 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d63039f-97ef-40e7-ada7-023397173098 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:03.070414 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d63039f-97ef-40e7-ada7-023397173098 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:03.070856 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d63039f-97ef-40e7-ada7-023397173098 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:03.116157 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac34a0a-129a-46a1-b704-5afc1de2bafd None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:03.143844 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d63039f-97ef-40e7-ada7-023397173098 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:03.145757 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:03.146310 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d63039f-97ef-40e7-ada7-023397173098 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:03.150653 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ae1d79-8f4f-43e9-ac29-b1b5661a0a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:03.151339 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ae1d79-8f4f-43e9-ac29-b1b5661a0a10 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:03.151691 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ae1d79-8f4f-43e9-ac29-b1b5661a0a10 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:03.152039 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ae1d79-8f4f-43e9-ac29-b1b5661a0a10 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:03.152835 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ae1d79-8f4f-43e9-ac29-b1b5661a0a10 None admin] RBAC: Adding query filter params (role.id=62f19ea272b447ccaab1576cd9bf4fd9, scope.project.id=0d32389226ec4d2dac8c390a33d5436c, user.id=e8b708e7077c461bb73c230d7b1825fb) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:03.153210 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ae1d79-8f4f-43e9-ac29-b1b5661a0a10 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:03.154854 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ae1d79-8f4f-43e9-ac29-b1b5661a0a10 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:03.160612 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 43/90] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Aug 25 03:00:03 2022] GET /identity/v3/role_assignments?user.id=e8b708e7077c461bb73c230d7b1825fb&scope.project.id=0d32389226ec4d2dac8c390a33d5436c&role.id=62f19ea272b447ccaab1576cd9bf4fd9 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:03.161536 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ae1d79-8f4f-43e9-ac29-b1b5661a0a10 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:03.266576 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 48/91] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Aug 25 03:00:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 25 03:00:03.267940 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d63039f-97ef-40e7-ada7-023397173098 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:03.269836 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ae1d79-8f4f-43e9-ac29-b1b5661a0a10 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:03.282666 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:03.306602 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c50d81-4d0d-4059-9592-ad6020b77d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:03.307329 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c50d81-4d0d-4059-9592-ad6020b77d1d None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:03.307682 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c50d81-4d0d-4059-9592-ad6020b77d1d None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:03.308055 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c50d81-4d0d-4059-9592-ad6020b77d1d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:03.308909 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c50d81-4d0d-4059-9592-ad6020b77d1d None admin] RBAC: Adding query filter params (role.id=3255f77192524a16bc405173106871fc, scope.project.id=0d32389226ec4d2dac8c390a33d5436c, user.id=26e7c9fc5d3c41de8156071d9f0cb3cd) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:03.309302 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c50d81-4d0d-4059-9592-ad6020b77d1d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:03.310956 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c50d81-4d0d-4059-9592-ad6020b77d1d None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:03.317626 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 44/92] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Aug 25 03:00:03 2022] GET /identity/v3/role_assignments?user.id=26e7c9fc5d3c41de8156071d9f0cb3cd&scope.project.id=0d32389226ec4d2dac8c390a33d5436c&role.id=3255f77192524a16bc405173106871fc => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:03.318418 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c50d81-4d0d-4059-9592-ad6020b77d1d None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:03.386838 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d63039f-97ef-40e7-ada7-023397173098 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:03.422714 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:03.456427 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8b1f55-32a1-4353-80a0-3ba9a9d06a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:03.457797 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8b1f55-32a1-4353-80a0-3ba9a9d06a4d None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:03.458261 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8b1f55-32a1-4353-80a0-3ba9a9d06a4d None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:03.458742 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8b1f55-32a1-4353-80a0-3ba9a9d06a4d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:03.490334 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c50d81-4d0d-4059-9592-ad6020b77d1d None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:03.534094 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6181a5-774a-4456-a668-08fcb916d240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:03.535839 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6181a5-774a-4456-a668-08fcb916d240 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:03.536105 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6181a5-774a-4456-a668-08fcb916d240 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:03.536495 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6181a5-774a-4456-a668-08fcb916d240 None admin] PATH_INFO: `/v3/projects/0d32389226ec4d2dac8c390a33d5436c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:03.537685 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6181a5-774a-4456-a668-08fcb916d240 None admin] RBAC: Authorizing `identity:get_project(project_id=0d32389226ec4d2dac8c390a33d5436c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:03.539278 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6181a5-774a-4456-a668-08fcb916d240 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:03.546307 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 45/93] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Aug 25 03:00:03 2022] GET /identity/v3/projects/0d32389226ec4d2dac8c390a33d5436c => generated 298 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:03.548220 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6181a5-774a-4456-a668-08fcb916d240 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:03.566424 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8b1f55-32a1-4353-80a0-3ba9a9d06a4d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:03.570614 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8b1f55-32a1-4353-80a0-3ba9a9d06a4d None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:03.602660 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:03.722465 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 49/94] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Aug 25 03:00:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 25 03:00:03.724772 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8b1f55-32a1-4353-80a0-3ba9a9d06a4d None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:03.745115 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6181a5-774a-4456-a668-08fcb916d240 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:03.790799 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:03.811813 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dcd26c-ef5b-438a-8bad-0e4f775e9f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:03.812448 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dcd26c-ef5b-438a-8bad-0e4f775e9f9d None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:03.812732 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dcd26c-ef5b-438a-8bad-0e4f775e9f9d None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:03.813011 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dcd26c-ef5b-438a-8bad-0e4f775e9f9d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:03.813752 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dcd26c-ef5b-438a-8bad-0e4f775e9f9d None admin] RBAC: Adding query filter params (role.id=7886cc7e333a42d08085932494a3ea6b, scope.project.id=0d32389226ec4d2dac8c390a33d5436c, user.id=26e7c9fc5d3c41de8156071d9f0cb3cd) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:03.814086 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dcd26c-ef5b-438a-8bad-0e4f775e9f9d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:03.815532 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dcd26c-ef5b-438a-8bad-0e4f775e9f9d None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:03.826320 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 46/95] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Aug 25 03:00:03 2022] GET /identity/v3/role_assignments?user.id=26e7c9fc5d3c41de8156071d9f0cb3cd&scope.project.id=0d32389226ec4d2dac8c390a33d5436c&role.id=7886cc7e333a42d08085932494a3ea6b => generated 259 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:00:03.847751 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8b1f55-32a1-4353-80a0-3ba9a9d06a4d None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:03.873584 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:03.898721 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265f3de0-5d59-4cf5-a9e7-39df6ed9f733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:03.899616 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265f3de0-5d59-4cf5-a9e7-39df6ed9f733 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:03.900073 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265f3de0-5d59-4cf5-a9e7-39df6ed9f733 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:03.900656 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265f3de0-5d59-4cf5-a9e7-39df6ed9f733 None admin] PATH_INFO: `/v3/projects/4a3808105deb490daa12dd1f297e4b06` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:03.901998 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265f3de0-5d59-4cf5-a9e7-39df6ed9f733 None admin] RBAC: Authorizing `identity:get_project(project_id=4a3808105deb490daa12dd1f297e4b06)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:03.903639 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265f3de0-5d59-4cf5-a9e7-39df6ed9f733 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:03.906310 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 50/96] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Aug 25 03:00:03 2022] GET /identity/v3/projects/4a3808105deb490daa12dd1f297e4b06 => generated 312 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:00:03.915476 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dcd26c-ef5b-438a-8bad-0e4f775e9f9d None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:04.060131 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dcd26c-ef5b-438a-8bad-0e4f775e9f9d None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:04.103873 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81135a14-68d5-4a82-9b02-a2423b3ef8fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:04.104498 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81135a14-68d5-4a82-9b02-a2423b3ef8fe None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:04.104779 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81135a14-68d5-4a82-9b02-a2423b3ef8fe None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:04.105052 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81135a14-68d5-4a82-9b02-a2423b3ef8fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:04.105792 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81135a14-68d5-4a82-9b02-a2423b3ef8fe None admin] RBAC: Adding query filter params (role.id=3255f77192524a16bc405173106871fc, scope.project.id=4a3808105deb490daa12dd1f297e4b06, user.id=26e7c9fc5d3c41de8156071d9f0cb3cd) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:04.106098 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81135a14-68d5-4a82-9b02-a2423b3ef8fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:04.107564 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81135a14-68d5-4a82-9b02-a2423b3ef8fe None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:04.118533 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 47/97] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Aug 25 03:00:03 2022] GET /identity/v3/role_assignments?user.id=26e7c9fc5d3c41de8156071d9f0cb3cd&scope.project.id=4a3808105deb490daa12dd1f297e4b06&role.id=3255f77192524a16bc405173106871fc => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:04.525587 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46967396-fd9a-45f8-9a25-11d8fde326d6 None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:04.526346 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46967396-fd9a-45f8-9a25-11d8fde326d6 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:04.526824 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46967396-fd9a-45f8-9a25-11d8fde326d6 None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:04.528426 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 51/98] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:00:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:00:04.533949 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80cb048-5942-4d1f-aa66-c5b50f043342 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:04.534507 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80cb048-5942-4d1f-aa66-c5b50f043342 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:04.535042 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80cb048-5942-4d1f-aa66-c5b50f043342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:04.553985 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-c80cb048-5942-4d1f-aa66-c5b50f043342 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:00:04.555530 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-c80cb048-5942-4d1f-aa66-c5b50f043342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:04.608954 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:04.625932 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 48/99] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:00:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 03:00:04.909143 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46967396-fd9a-45f8-9a25-11d8fde326d6 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:05.083901 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46967396-fd9a-45f8-9a25-11d8fde326d6 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:05.117787 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01d6dad-7c88-48e8-aa4a-79f63c17e9ac None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:05.118343 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01d6dad-7c88-48e8-aa4a-79f63c17e9ac None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:05.118343 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01d6dad-7c88-48e8-aa4a-79f63c17e9ac None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:05.119856 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 49/100] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:00:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:00:05.125234 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a914fb8-ead7-4b66-a212-52676e229f04 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:05.125518 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a914fb8-ead7-4b66-a212-52676e229f04 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:05.125794 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a914fb8-ead7-4b66-a212-52676e229f04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:05.156113 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc9d24c-fb73-4bf7-91f6-8093a81bd6c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:05.156823 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc9d24c-fb73-4bf7-91f6-8093a81bd6c6 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:05.157109 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc9d24c-fb73-4bf7-91f6-8093a81bd6c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:05.157382 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc9d24c-fb73-4bf7-91f6-8093a81bd6c6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:05.161128 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-6a914fb8-ead7-4b66-a212-52676e229f04 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:00:05.161413 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc9d24c-fb73-4bf7-91f6-8093a81bd6c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:05.162908 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc9d24c-fb73-4bf7-91f6-8093a81bd6c6 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:05.165719 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-6a914fb8-ead7-4b66-a212-52676e229f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:05.169362 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 52/101] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:00:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:05.194943 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc9d24c-fb73-4bf7-91f6-8093a81bd6c6 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:05.264152 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:05.334379 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc9d24c-fb73-4bf7-91f6-8093a81bd6c6 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:05.361394 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 50/102] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:00:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 03:00:05.378144 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83943dee-52c2-470b-bcd4-f27369331b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:05.378831 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83943dee-52c2-470b-bcd4-f27369331b3f None admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:05.379118 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83943dee-52c2-470b-bcd4-f27369331b3f None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:05.379392 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83943dee-52c2-470b-bcd4-f27369331b3f None admin] PATH_INFO: `/v3/projects` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:05.380064 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83943dee-52c2-470b-bcd4-f27369331b3f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:05.381582 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83943dee-52c2-470b-bcd4-f27369331b3f None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:05.491466 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 53/103] 10.222.0.58 () {60 vars in 1141 bytes} [Thu Aug 25 03:00:05 2022] POST /identity/v3/projects => generated 302 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:00:05.881769 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a914fb8-ead7-4b66-a212-52676e229f04 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:06.012318 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a914fb8-ead7-4b66-a212-52676e229f04 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:06.031099 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:06.057312 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14df2383-b1b9-4ae3-bd08-6cd2ae456f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:06.058344 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14df2383-b1b9-4ae3-bd08-6cd2ae456f33 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:06.058831 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14df2383-b1b9-4ae3-bd08-6cd2ae456f33 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:06.059403 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14df2383-b1b9-4ae3-bd08-6cd2ae456f33 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:06.078918 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14df2383-b1b9-4ae3-bd08-6cd2ae456f33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:00:06.084136 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-14df2383-b1b9-4ae3-bd08-6cd2ae456f33 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 25 03:00:06.084136 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:00:06.084136 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:00:06.084136 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:00:06.084136 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:00:06.084136 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:00:06.084136 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:00:06.084136 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:00:06.084136 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:00:06.084136 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:00:06.084136 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:00:06.084136 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:00:06.084136 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:00:06.084136 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:00:06.084136 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:00:06.084136 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:00:06.084981 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:00:06.084981 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:00:06.084981 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:00:06.084981 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:00:06.084981 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:00:06.084981 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:00:06.084981 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:00:06.084981 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:00:06.084981 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:00:06.084981 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:00:06.084981 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:00:06.084981 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:00:06.084981 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:00:06.084981 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:00:06.084981 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:00:06.084981 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:00:06.085716 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:00:06.085716 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:00:06.085716 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:00:06.085716 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:00:06.085716 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:00:06.085716 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:00:06.085716 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:00:06.085716 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:00:06.085716 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:00:06.085716 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:00:06.085716 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 25 03:00:06.085716 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:00:06.088006 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 51/104] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Aug 25 03:00:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:00:06.094034 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83943dee-52c2-470b-bcd4-f27369331b3f None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:06.250269 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:06.257636 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83943dee-52c2-470b-bcd4-f27369331b3f None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:06.303389 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38516fe9-5bd4-40b4-a673-2d5b509e7bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:06.304023 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38516fe9-5bd4-40b4-a673-2d5b509e7bac None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:06.304307 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38516fe9-5bd4-40b4-a673-2d5b509e7bac None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:06.304589 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38516fe9-5bd4-40b4-a673-2d5b509e7bac None admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:06.305276 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38516fe9-5bd4-40b4-a673-2d5b509e7bac None admin] RBAC: Adding query filter params (name=reader) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:06.305587 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38516fe9-5bd4-40b4-a673-2d5b509e7bac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:06.307048 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38516fe9-5bd4-40b4-a673-2d5b509e7bac None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:06.323018 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 54/105] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Aug 25 03:00:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:06.330011 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14df2383-b1b9-4ae3-bd08-6cd2ae456f33 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:06.408234 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b43bb2-0a8f-4653-a977-cadabc530612 None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:06.408566 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b43bb2-0a8f-4653-a977-cadabc530612 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:06.408798 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b43bb2-0a8f-4653-a977-cadabc530612 None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:06.410703 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 55/106] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:00:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:00:06.422858 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5824a991-1a60-46fd-b089-e86454d01cb1 None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:06.423136 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5824a991-1a60-46fd-b089-e86454d01cb1 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:06.423486 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5824a991-1a60-46fd-b089-e86454d01cb1 None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:06.427243 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 56/107] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:00:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:00:06.428673 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae86b05a-fbc1-43f3-8b35-55b72dc56ef3 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:06.428961 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae86b05a-fbc1-43f3-8b35-55b72dc56ef3 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:06.429245 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae86b05a-fbc1-43f3-8b35-55b72dc56ef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:06.463259 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-ae86b05a-fbc1-43f3-8b35-55b72dc56ef3 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:00:06.464470 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-ae86b05a-fbc1-43f3-8b35-55b72dc56ef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:06.485405 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14df2383-b1b9-4ae3-bd08-6cd2ae456f33 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:06.535912 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b2d8a5-7037-4fb9-86ed-633896462ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:06.536249 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b2d8a5-7037-4fb9-86ed-633896462ad8 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:06.536536 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b2d8a5-7037-4fb9-86ed-633896462ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:06.536815 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b2d8a5-7037-4fb9-86ed-633896462ad8 None admin] PATH_INFO: `/v3/users/064052b980324793a0bc0266db01c33b` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:06.562998 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b2d8a5-7037-4fb9-86ed-633896462ad8 None admin] RBAC: Authorizing `identity:get_user(user_id=064052b980324793a0bc0266db01c33b)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:06.564517 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b2d8a5-7037-4fb9-86ed-633896462ad8 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:06.566238 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 52/108] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Aug 25 03:00:06 2022] GET /identity/v3/users/064052b980324793a0bc0266db01c33b => generated 289 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:06.566980 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 57/109] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:00:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 03:00:06.568638 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfa7fc8-6858-47dc-85ec-2e315d648771 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:06.568939 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfa7fc8-6858-47dc-85ec-2e315d648771 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:06.569255 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfa7fc8-6858-47dc-85ec-2e315d648771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:06.575744 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae86b05a-fbc1-43f3-8b35-55b72dc56ef3 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:06.618532 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-bbfa7fc8-6858-47dc-85ec-2e315d648771 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:00:06.619754 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-bbfa7fc8-6858-47dc-85ec-2e315d648771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:06.658983 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:06.683759 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:06.700826 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae86b05a-fbc1-43f3-8b35-55b72dc56ef3 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:06.740546 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ee0e36-e54d-4566-87fd-ab145129bc6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:06.740951 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 53/110] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:00:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 03:00:06.741719 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ee0e36-e54d-4566-87fd-ab145129bc6c None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:06.742224 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ee0e36-e54d-4566-87fd-ab145129bc6c None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:06.742528 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be5f71d-7c66-41df-89ff-2faecda0de9a None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:06.742676 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be5f71d-7c66-41df-89ff-2faecda0de9a None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:06.742941 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be5f71d-7c66-41df-89ff-2faecda0de9a None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:06.743324 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ee0e36-e54d-4566-87fd-ab145129bc6c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:06.744193 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 54/111] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:00:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:00:06.746401 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe80951-a5d5-42d9-ae1e-a50687a87793 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:06.746691 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe80951-a5d5-42d9-ae1e-a50687a87793 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:06.747053 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe80951-a5d5-42d9-ae1e-a50687a87793 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:06.749463 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 55/112] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:00:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:00:06.753162 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8207be60-bfec-433b-aec0-1ada0cf643b0 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:06.753456 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8207be60-bfec-433b-aec0-1ada0cf643b0 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:06.753739 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8207be60-bfec-433b-aec0-1ada0cf643b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:06.779494 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-8207be60-bfec-433b-aec0-1ada0cf643b0 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:00:06.780719 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-8207be60-bfec-433b-aec0-1ada0cf643b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:06.822150 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ee0e36-e54d-4566-87fd-ab145129bc6c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:06.824209 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ee0e36-e54d-4566-87fd-ab145129bc6c None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:06.908564 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 56/113] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:00:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 03:00:06.909899 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a320e9-3fbb-4673-8a84-ad1401c741cf None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:06.910205 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a320e9-3fbb-4673-8a84-ad1401c741cf None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:06.910524 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a320e9-3fbb-4673-8a84-ad1401c741cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:06.915208 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:06.932926 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 58/114] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Aug 25 03:00:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 25 03:00:06.941214 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ee0e36-e54d-4566-87fd-ab145129bc6c None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:06.955893 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-b1a320e9-3fbb-4673-8a84-ad1401c741cf None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:00:06.960928 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-b1a320e9-3fbb-4673-8a84-ad1401c741cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:07.063687 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 57/115] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:00:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 03:00:07.064386 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a320e9-3fbb-4673-8a84-ad1401c741cf None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:07.115987 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ee0e36-e54d-4566-87fd-ab145129bc6c None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:07.138664 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:07.163977 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c94d61-df0a-401f-bd52-c79effd12205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:07.164990 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c94d61-df0a-401f-bd52-c79effd12205 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:07.166520 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c94d61-df0a-401f-bd52-c79effd12205 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:07.167005 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c94d61-df0a-401f-bd52-c79effd12205 None admin] PATH_INFO: `/v3/projects/0d32389226ec4d2dac8c390a33d5436c` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:07.169140 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c94d61-df0a-401f-bd52-c79effd12205 None admin] RBAC: Authorizing `identity:get_project(project_id=0d32389226ec4d2dac8c390a33d5436c)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:07.171098 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c94d61-df0a-401f-bd52-c79effd12205 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:07.173620 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 59/116] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Aug 25 03:00:06 2022] GET /identity/v3/projects/0d32389226ec4d2dac8c390a33d5436c => generated 298 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:00:07.174814 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c94d61-df0a-401f-bd52-c79effd12205 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:07.250450 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a320e9-3fbb-4673-8a84-ad1401c741cf None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:07.272356 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:07.301290 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72bd082-02a1-4f53-9d53-dc2494268630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:07.302861 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72bd082-02a1-4f53-9d53-dc2494268630 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:07.303385 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72bd082-02a1-4f53-9d53-dc2494268630 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:07.303882 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72bd082-02a1-4f53-9d53-dc2494268630 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:07.319121 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72bd082-02a1-4f53-9d53-dc2494268630 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:00:07.326043 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c94d61-df0a-401f-bd52-c79effd12205 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:07.326374 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-d72bd082-02a1-4f53-9d53-dc2494268630 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 25 03:00:07.326374 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:00:07.326374 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:00:07.326374 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:00:07.326374 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:00:07.326374 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:00:07.326374 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:00:07.326374 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:00:07.326374 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:00:07.326374 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:00:07.326374 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:00:07.326374 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:00:07.326374 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:00:07.326374 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:00:07.326374 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:00:07.326374 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:00:07.327505 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:00:07.327505 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:00:07.327505 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:00:07.327505 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:00:07.327505 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:00:07.327505 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:00:07.327505 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:00:07.327505 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:00:07.327505 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:00:07.327505 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:00:07.327505 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:00:07.327505 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:00:07.327505 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:00:07.327505 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:00:07.327505 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:00:07.327505 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:00:07.328246 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:00:07.328246 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:00:07.328246 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:00:07.328246 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:00:07.328246 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:00:07.328246 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:00:07.328246 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:00:07.328246 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:00:07.328246 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:00:07.328246 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:00:07.328246 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 25 03:00:07.328246 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:00:07.331713 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 58/117] 10.222.0.58 () {56 vars in 1114 bytes} [Thu Aug 25 03:00:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:00:07.333164 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72bd082-02a1-4f53-9d53-dc2494268630 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:07.374023 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4249d515-57ca-495a-9c38-9ea2d9eff0fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:07.375007 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4249d515-57ca-495a-9c38-9ea2d9eff0fd None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:07.375523 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4249d515-57ca-495a-9c38-9ea2d9eff0fd None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:07.375999 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4249d515-57ca-495a-9c38-9ea2d9eff0fd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:07.387061 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4249d515-57ca-495a-9c38-9ea2d9eff0fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:00:07.393224 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-4249d515-57ca-495a-9c38-9ea2d9eff0fd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 03:00:07.393224 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:00:07.393224 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:00:07.393224 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:00:07.393224 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:00:07.393224 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:00:07.393224 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:00:07.393224 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:00:07.393224 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:00:07.393224 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:00:07.393224 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:00:07.393224 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:00:07.393224 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:00:07.393224 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:00:07.393224 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:00:07.393224 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:00:07.394089 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:00:07.394089 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:00:07.394089 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:00:07.394089 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:00:07.394089 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:00:07.394089 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:00:07.394089 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:00:07.394089 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:00:07.394089 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:00:07.394089 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:00:07.394089 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:00:07.394089 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:00:07.394089 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:00:07.394089 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:00:07.394089 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:00:07.394089 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:00:07.394863 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:00:07.394863 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:00:07.394863 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:00:07.394863 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:00:07.394863 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:00:07.394863 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:00:07.394863 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:00:07.394863 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:00:07.394863 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:00:07.394863 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:00:07.394863 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 03:00:07.394863 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:00:07.395933 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 60/118] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Aug 25 03:00:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:00:07.396349 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4249d515-57ca-495a-9c38-9ea2d9eff0fd None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:07.433618 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:07.448999 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72bd082-02a1-4f53-9d53-dc2494268630 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:07.486442 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30c71da-ae29-4f9f-ba98-b6f15697d617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:07.487117 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30c71da-ae29-4f9f-ba98-b6f15697d617 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:07.487405 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30c71da-ae29-4f9f-ba98-b6f15697d617 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:07.487686 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30c71da-ae29-4f9f-ba98-b6f15697d617 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:07.488450 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30c71da-ae29-4f9f-ba98-b6f15697d617 None admin] RBAC: Adding query filter params (role.id=3de9d2fd0b464c399f67fb5240905596, scope.project.id=0d32389226ec4d2dac8c390a33d5436c, user.id=064052b980324793a0bc0266db01c33b) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:07.488768 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30c71da-ae29-4f9f-ba98-b6f15697d617 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:07.490233 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30c71da-ae29-4f9f-ba98-b6f15697d617 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:07.496125 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 59/119] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Aug 25 03:00:07 2022] GET /identity/v3/role_assignments?user.id=064052b980324793a0bc0266db01c33b&scope.project.id=0d32389226ec4d2dac8c390a33d5436c&role.id=3de9d2fd0b464c399f67fb5240905596 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:07.496912 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30c71da-ae29-4f9f-ba98-b6f15697d617 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:07.516888 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4249d515-57ca-495a-9c38-9ea2d9eff0fd None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:07.537921 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:07.569096 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1405550-6dc1-4b6c-b34b-0968dda94785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:07.570273 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1405550-6dc1-4b6c-b34b-0968dda94785 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:07.570956 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1405550-6dc1-4b6c-b34b-0968dda94785 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:07.571449 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1405550-6dc1-4b6c-b34b-0968dda94785 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:07.592312 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1405550-6dc1-4b6c-b34b-0968dda94785 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:00:07.601446 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-d1405550-6dc1-4b6c-b34b-0968dda94785 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 25 03:00:07.601446 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:00:07.601446 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:00:07.601446 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:00:07.601446 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:00:07.601446 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:00:07.601446 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:00:07.601446 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:00:07.601446 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:00:07.601446 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:00:07.601446 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:00:07.601446 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:00:07.601446 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:00:07.601446 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:00:07.601446 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:00:07.601446 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:00:07.602397 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:00:07.602397 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:00:07.602397 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:00:07.602397 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:00:07.602397 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:00:07.602397 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:00:07.602397 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:00:07.602397 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:00:07.602397 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:00:07.602397 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:00:07.602397 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:00:07.602397 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:00:07.602397 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:00:07.602397 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:00:07.602397 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:00:07.602397 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:00:07.603229 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:00:07.603229 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:00:07.603229 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:00:07.603229 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:00:07.603229 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:00:07.603229 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:00:07.603229 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:00:07.603229 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:00:07.603229 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:00:07.603229 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:00:07.603229 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 25 03:00:07.603229 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:00:07.605773 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 61/120] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Aug 25 03:00:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:00:07.607559 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1405550-6dc1-4b6c-b34b-0968dda94785 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:07.644778 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30c71da-ae29-4f9f-ba98-b6f15697d617 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:07.688223 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78b5c6a-174e-401c-839f-356f02fa1403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:07.689217 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78b5c6a-174e-401c-839f-356f02fa1403 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:07.689507 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78b5c6a-174e-401c-839f-356f02fa1403 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:07.689855 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78b5c6a-174e-401c-839f-356f02fa1403 None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:07.690979 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78b5c6a-174e-401c-839f-356f02fa1403 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:07.691293 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78b5c6a-174e-401c-839f-356f02fa1403 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:07.693214 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78b5c6a-174e-401c-839f-356f02fa1403 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:07.705118 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 60/121] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Aug 25 03:00:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:07.707162 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78b5c6a-174e-401c-839f-356f02fa1403 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:07.724089 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:07.750011 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1405550-6dc1-4b6c-b34b-0968dda94785 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:07.795167 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed955e4-eef6-4221-9fe9-536e51e15ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:07.796232 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed955e4-eef6-4221-9fe9-536e51e15ba2 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:07.796718 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed955e4-eef6-4221-9fe9-536e51e15ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:07.797196 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed955e4-eef6-4221-9fe9-536e51e15ba2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:07.812284 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed955e4-eef6-4221-9fe9-536e51e15ba2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:00:07.819492 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-8ed955e4-eef6-4221-9fe9-536e51e15ba2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 25 03:00:07.819492 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:00:07.819492 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:00:07.819492 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:00:07.819492 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:00:07.819492 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:00:07.819492 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:00:07.819492 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:00:07.819492 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:00:07.819492 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:00:07.819492 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:00:07.819492 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:00:07.819492 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:00:07.819492 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:00:07.819492 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:00:07.819492 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:00:07.820356 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:00:07.820356 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:00:07.820356 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:00:07.820356 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:00:07.820356 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:00:07.820356 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:00:07.820356 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:00:07.820356 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:00:07.820356 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:00:07.820356 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:00:07.820356 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:00:07.820356 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:00:07.820356 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:00:07.820356 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:00:07.820356 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:00:07.820356 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:00:07.821143 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:00:07.821143 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:00:07.821143 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:00:07.821143 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:00:07.821143 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:00:07.821143 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:00:07.821143 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:00:07.821143 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:00:07.821143 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:00:07.821143 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:00:07.821143 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 25 03:00:07.821143 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:00:07.821720 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 62/122] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Aug 25 03:00:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:00:07.824158 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed955e4-eef6-4221-9fe9-536e51e15ba2 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:07.840717 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78b5c6a-174e-401c-839f-356f02fa1403 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:07.877390 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a22ee47-1afb-4144-aa3e-54ca8bc65941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:07.877845 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a22ee47-1afb-4144-aa3e-54ca8bc65941 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:07.878126 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a22ee47-1afb-4144-aa3e-54ca8bc65941 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:07.878446 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a22ee47-1afb-4144-aa3e-54ca8bc65941 None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:07.879133 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a22ee47-1afb-4144-aa3e-54ca8bc65941 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:07.879421 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a22ee47-1afb-4144-aa3e-54ca8bc65941 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:07.880794 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a22ee47-1afb-4144-aa3e-54ca8bc65941 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:07.882117 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:07.894716 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 61/123] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Aug 25 03:00:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:07.895611 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a22ee47-1afb-4144-aa3e-54ca8bc65941 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:07.973156 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:07.977116 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed955e4-eef6-4221-9fe9-536e51e15ba2 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:08.005067 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a22ee47-1afb-4144-aa3e-54ca8bc65941 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:08.023428 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b703e4d3-980d-430d-93b6-f02554c23fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:08.024338 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b703e4d3-980d-430d-93b6-f02554c23fc4 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:08.024801 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b703e4d3-980d-430d-93b6-f02554c23fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:08.025256 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b703e4d3-980d-430d-93b6-f02554c23fc4 None admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:08.026675 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b703e4d3-980d-430d-93b6-f02554c23fc4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:08.027206 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b703e4d3-980d-430d-93b6-f02554c23fc4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:08.029261 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b703e4d3-980d-430d-93b6-f02554c23fc4 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:08.042111 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 63/124] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Aug 25 03:00:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:08.047218 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b703e4d3-980d-430d-93b6-f02554c23fc4 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:08.051870 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4b8522-2c2d-45e4-a59a-0343dbb9a3ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:08.052758 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4b8522-2c2d-45e4-a59a-0343dbb9a3ff None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:08.053220 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4b8522-2c2d-45e4-a59a-0343dbb9a3ff None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:08.053669 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4b8522-2c2d-45e4-a59a-0343dbb9a3ff None admin] PATH_INFO: `/v3/users/26e7c9fc5d3c41de8156071d9f0cb3cd` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:08.055048 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4b8522-2c2d-45e4-a59a-0343dbb9a3ff None admin] RBAC: Authorizing `identity:get_user(user_id=26e7c9fc5d3c41de8156071d9f0cb3cd)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:08.057317 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4b8522-2c2d-45e4-a59a-0343dbb9a3ff None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:08.059721 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 62/125] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Aug 25 03:00:07 2022] GET /identity/v3/users/26e7c9fc5d3c41de8156071d9f0cb3cd => generated 275 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:08.060687 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4b8522-2c2d-45e4-a59a-0343dbb9a3ff None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:08.151854 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:08.158909 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b703e4d3-980d-430d-93b6-f02554c23fc4 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:08.178623 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4b8522-2c2d-45e4-a59a-0343dbb9a3ff None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:08.195654 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb4ba8d-1c19-4f63-a4f1-f056342d4894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:08.196264 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb4ba8d-1c19-4f63-a4f1-f056342d4894 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:08.196579 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb4ba8d-1c19-4f63-a4f1-f056342d4894 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:08.197108 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb4ba8d-1c19-4f63-a4f1-f056342d4894 None admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:08.197971 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb4ba8d-1c19-4f63-a4f1-f056342d4894 None admin] RBAC: Adding query filter params (name=member) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:08.198607 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb4ba8d-1c19-4f63-a4f1-f056342d4894 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:08.200001 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb4ba8d-1c19-4f63-a4f1-f056342d4894 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:08.209055 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 64/126] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Aug 25 03:00:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:08.209830 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb4ba8d-1c19-4f63-a4f1-f056342d4894 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:08.219422 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b1604b-c161-4bd3-8be0-a35db723aa69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:08.220072 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b1604b-c161-4bd3-8be0-a35db723aa69 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:08.220362 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b1604b-c161-4bd3-8be0-a35db723aa69 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:08.220645 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b1604b-c161-4bd3-8be0-a35db723aa69 None admin] PATH_INFO: `/v3/users/e8b708e7077c461bb73c230d7b1825fb` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:08.221770 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b1604b-c161-4bd3-8be0-a35db723aa69 None admin] RBAC: Authorizing `identity:get_user(user_id=e8b708e7077c461bb73c230d7b1825fb)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:08.223277 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b1604b-c161-4bd3-8be0-a35db723aa69 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:08.229482 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 63/127] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Aug 25 03:00:08 2022] GET /identity/v3/users/e8b708e7077c461bb73c230d7b1825fb => generated 247 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:08.229482 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b1604b-c161-4bd3-8be0-a35db723aa69 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:08.239544 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:08.323524 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb4ba8d-1c19-4f63-a4f1-f056342d4894 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:08.328059 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b1604b-c161-4bd3-8be0-a35db723aa69 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:08.359115 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f99e04-4464-4cba-8a02-e92958e9f0cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:08.359751 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f99e04-4464-4cba-8a02-e92958e9f0cc None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:08.360126 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f99e04-4464-4cba-8a02-e92958e9f0cc None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:08.360503 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f99e04-4464-4cba-8a02-e92958e9f0cc None admin] PATH_INFO: `/v3/users/26e7c9fc5d3c41de8156071d9f0cb3cd` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:08.361376 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2044b6a-33b3-4b69-839c-9c0a919b84a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:08.361806 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f99e04-4464-4cba-8a02-e92958e9f0cc None admin] RBAC: Authorizing `identity:get_user(user_id=26e7c9fc5d3c41de8156071d9f0cb3cd)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:08.362018 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2044b6a-33b3-4b69-839c-9c0a919b84a6 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:08.362326 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2044b6a-33b3-4b69-839c-9c0a919b84a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:08.362615 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2044b6a-33b3-4b69-839c-9c0a919b84a6 None admin] PATH_INFO: `/v3/projects/0d32389226ec4d2dac8c390a33d5436c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:08.363197 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f99e04-4464-4cba-8a02-e92958e9f0cc None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:08.363736 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2044b6a-33b3-4b69-839c-9c0a919b84a6 None admin] RBAC: Authorizing `identity:get_project(project_id=0d32389226ec4d2dac8c390a33d5436c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:08.365070 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 65/128] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Aug 25 03:00:08 2022] GET /identity/v3/users/26e7c9fc5d3c41de8156071d9f0cb3cd => generated 275 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:08.365165 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2044b6a-33b3-4b69-839c-9c0a919b84a6 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:08.366025 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697717dd-02af-4be0-bedd-d65872fd176f None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:08.370329 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697717dd-02af-4be0-bedd-d65872fd176f None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:08.370329 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697717dd-02af-4be0-bedd-d65872fd176f None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:08.370528 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 64/129] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Aug 25 03:00:08 2022] GET /identity/v3/projects/0d32389226ec4d2dac8c390a33d5436c => generated 298 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:08.370528 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2044b6a-33b3-4b69-839c-9c0a919b84a6 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:08.372527 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 66/130] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:00:08 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:00:08.373137 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697717dd-02af-4be0-bedd-d65872fd176f None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:08.407311 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:08.450824 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:08.475851 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2044b6a-33b3-4b69-839c-9c0a919b84a6 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:08.483005 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697717dd-02af-4be0-bedd-d65872fd176f None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:08.515050 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c07a3d-2abf-48a1-90ae-cf2f503d4235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:08.515696 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c07a3d-2abf-48a1-90ae-cf2f503d4235 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:08.515990 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c07a3d-2abf-48a1-90ae-cf2f503d4235 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:08.516264 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c07a3d-2abf-48a1-90ae-cf2f503d4235 None admin] PATH_INFO: `/v3/users/26e7c9fc5d3c41de8156071d9f0cb3cd` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:08.517363 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c07a3d-2abf-48a1-90ae-cf2f503d4235 None admin] RBAC: Authorizing `identity:get_user(user_id=26e7c9fc5d3c41de8156071d9f0cb3cd)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:08.518877 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c07a3d-2abf-48a1-90ae-cf2f503d4235 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:08.519914 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b2e845-f898-4a3f-b64e-c306862cf5bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:08.520633 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 65/131] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Aug 25 03:00:08 2022] GET /identity/v3/users/26e7c9fc5d3c41de8156071d9f0cb3cd => generated 275 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:08.520734 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b2e845-f898-4a3f-b64e-c306862cf5bc None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:08.520815 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b2e845-f898-4a3f-b64e-c306862cf5bc None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:08.521072 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b2e845-f898-4a3f-b64e-c306862cf5bc None admin] PATH_INFO: `/v3/projects/0d32389226ec4d2dac8c390a33d5436c` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:08.522189 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b2e845-f898-4a3f-b64e-c306862cf5bc None admin] RBAC: Authorizing `identity:get_project(project_id=0d32389226ec4d2dac8c390a33d5436c)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:08.523659 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b2e845-f898-4a3f-b64e-c306862cf5bc None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:08.525290 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 67/132] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Aug 25 03:00:08 2022] GET /identity/v3/projects/0d32389226ec4d2dac8c390a33d5436c => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:08.525890 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c07a3d-2abf-48a1-90ae-cf2f503d4235 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:08.526046 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b2e845-f898-4a3f-b64e-c306862cf5bc None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:08.640560 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:08.641317 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c07a3d-2abf-48a1-90ae-cf2f503d4235 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:08.648158 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b2e845-f898-4a3f-b64e-c306862cf5bc None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:08.675963 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd5d673-1f9a-4159-8ae1-72f07d186fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:08.676527 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd5d673-1f9a-4159-8ae1-72f07d186fa1 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:08.676817 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd5d673-1f9a-4159-8ae1-72f07d186fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:08.677096 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd5d673-1f9a-4159-8ae1-72f07d186fa1 None admin] PATH_INFO: `/v3/projects/0d32389226ec4d2dac8c390a33d5436c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:08.678224 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd5d673-1f9a-4159-8ae1-72f07d186fa1 None admin] RBAC: Authorizing `identity:get_project(project_id=0d32389226ec4d2dac8c390a33d5436c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:08.679711 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd5d673-1f9a-4159-8ae1-72f07d186fa1 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:08.685928 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 66/133] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Aug 25 03:00:08 2022] GET /identity/v3/projects/0d32389226ec4d2dac8c390a33d5436c => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:08.685928 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0195d46-4a0b-4ba1-95b4-01d0616a014b None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:08.685928 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0195d46-4a0b-4ba1-95b4-01d0616a014b None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:08.685928 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0195d46-4a0b-4ba1-95b4-01d0616a014b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:08.693804 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ada03c-5624-47b2-9afb-7d3bb93ade18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:08.694512 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ada03c-5624-47b2-9afb-7d3bb93ade18 None admin] REQUEST_METHOD: `PUT` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:08.694851 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ada03c-5624-47b2-9afb-7d3bb93ade18 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:08.695062 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ada03c-5624-47b2-9afb-7d3bb93ade18 None admin] PATH_INFO: `/v3/projects/0d32389226ec4d2dac8c390a33d5436c/users/26e7c9fc5d3c41de8156071d9f0cb3cd/roles/7886cc7e333a42d08085932494a3ea6b` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:08.697255 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ada03c-5624-47b2-9afb-7d3bb93ade18 None admin] RBAC: Authorizing `identity:create_grant(project_id=0d32389226ec4d2dac8c390a33d5436c, user_id=26e7c9fc5d3c41de8156071d9f0cb3cd, role_id=7886cc7e333a42d08085932494a3ea6b)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:08.698737 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ada03c-5624-47b2-9afb-7d3bb93ade18 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:08.703822 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-e0195d46-4a0b-4ba1-95b4-01d0616a014b None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:00:08.705002 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-e0195d46-4a0b-4ba1-95b4-01d0616a014b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:08.712504 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:08.770353 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 67/134] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:00:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 03:00:08.771498 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 68/135] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Aug 25 03:00:08 2022] PUT /identity/v3/projects/0d32389226ec4d2dac8c390a33d5436c/users/26e7c9fc5d3c41de8156071d9f0cb3cd/roles/7886cc7e333a42d08085932494a3ea6b => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:00:08.771745 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0195d46-4a0b-4ba1-95b4-01d0616a014b None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:08.772256 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ada03c-5624-47b2-9afb-7d3bb93ade18 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:08.893860 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0195d46-4a0b-4ba1-95b4-01d0616a014b None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:08.928832 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ada03c-5624-47b2-9afb-7d3bb93ade18 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:08.934260 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6805e5-f18d-408b-9bb0-9f689688d803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:08.935222 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6805e5-f18d-408b-9bb0-9f689688d803 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:08.935657 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6805e5-f18d-408b-9bb0-9f689688d803 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:08.936088 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6805e5-f18d-408b-9bb0-9f689688d803 None admin] PATH_INFO: `/v3/projects/4a3808105deb490daa12dd1f297e4b06` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:08.938098 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6805e5-f18d-408b-9bb0-9f689688d803 None admin] RBAC: Authorizing `identity:get_project(project_id=4a3808105deb490daa12dd1f297e4b06)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:08.939968 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6805e5-f18d-408b-9bb0-9f689688d803 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:08.942032 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 68/136] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Aug 25 03:00:08 2022] GET /identity/v3/projects/4a3808105deb490daa12dd1f297e4b06 => generated 312 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:08.943365 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6805e5-f18d-408b-9bb0-9f689688d803 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:08.959669 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:08.982144 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d1d16f-a5a7-40f8-ad08-d0dcfaaa20fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:08.984074 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d1d16f-a5a7-40f8-ad08-d0dcfaaa20fe None admin] REQUEST_METHOD: `PUT` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:08.984526 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d1d16f-a5a7-40f8-ad08-d0dcfaaa20fe None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:08.985079 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d1d16f-a5a7-40f8-ad08-d0dcfaaa20fe None admin] PATH_INFO: `/v3/projects/0d32389226ec4d2dac8c390a33d5436c/users/e8b708e7077c461bb73c230d7b1825fb/roles/62f19ea272b447ccaab1576cd9bf4fd9` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:08.988229 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d1d16f-a5a7-40f8-ad08-d0dcfaaa20fe None admin] RBAC: Authorizing `identity:create_grant(project_id=0d32389226ec4d2dac8c390a33d5436c, user_id=e8b708e7077c461bb73c230d7b1825fb, role_id=62f19ea272b447ccaab1576cd9bf4fd9)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:08.989723 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d1d16f-a5a7-40f8-ad08-d0dcfaaa20fe None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:09.073977 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6805e5-f18d-408b-9bb0-9f689688d803 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:09.087583 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 69/137] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Aug 25 03:00:08 2022] PUT /identity/v3/projects/0d32389226ec4d2dac8c390a33d5436c/users/e8b708e7077c461bb73c230d7b1825fb/roles/62f19ea272b447ccaab1576cd9bf4fd9 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:00:09.088908 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d1d16f-a5a7-40f8-ad08-d0dcfaaa20fe None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:09.125451 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:09.129555 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3074fa32-4428-4ca5-9409-1322a6c1bfc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:09.130267 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3074fa32-4428-4ca5-9409-1322a6c1bfc5 None admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:09.130576 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3074fa32-4428-4ca5-9409-1322a6c1bfc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:09.130856 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3074fa32-4428-4ca5-9409-1322a6c1bfc5 None admin] PATH_INFO: `/v3/projects/0d32389226ec4d2dac8c390a33d5436c/users/26e7c9fc5d3c41de8156071d9f0cb3cd/roles/3255f77192524a16bc405173106871fc` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:09.135955 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3074fa32-4428-4ca5-9409-1322a6c1bfc5 None admin] RBAC: Authorizing `identity:create_grant(project_id=0d32389226ec4d2dac8c390a33d5436c, user_id=26e7c9fc5d3c41de8156071d9f0cb3cd, role_id=3255f77192524a16bc405173106871fc)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:09.137753 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3074fa32-4428-4ca5-9409-1322a6c1bfc5 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:09.228511 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 69/138] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Aug 25 03:00:08 2022] PUT /identity/v3/projects/0d32389226ec4d2dac8c390a33d5436c/users/26e7c9fc5d3c41de8156071d9f0cb3cd/roles/3255f77192524a16bc405173106871fc => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:00:09.229116 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3074fa32-4428-4ca5-9409-1322a6c1bfc5 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:09.265237 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d1d16f-a5a7-40f8-ad08-d0dcfaaa20fe None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:09.310196 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac7d004-c8e4-42ed-ab50-167785f37650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:09.310932 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac7d004-c8e4-42ed-ab50-167785f37650 None admin] REQUEST_METHOD: `PUT` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:09.311252 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac7d004-c8e4-42ed-ab50-167785f37650 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:09.311560 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac7d004-c8e4-42ed-ab50-167785f37650 None admin] PATH_INFO: `/v3/projects/4a3808105deb490daa12dd1f297e4b06/users/26e7c9fc5d3c41de8156071d9f0cb3cd/roles/3255f77192524a16bc405173106871fc` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:09.316886 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac7d004-c8e4-42ed-ab50-167785f37650 None admin] RBAC: Authorizing `identity:create_grant(project_id=4a3808105deb490daa12dd1f297e4b06, user_id=26e7c9fc5d3c41de8156071d9f0cb3cd, role_id=3255f77192524a16bc405173106871fc)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:09.318428 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac7d004-c8e4-42ed-ab50-167785f37650 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:09.369186 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:09.400985 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 70/139] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Aug 25 03:00:09 2022] PUT /identity/v3/projects/4a3808105deb490daa12dd1f297e4b06/users/26e7c9fc5d3c41de8156071d9f0cb3cd/roles/3255f77192524a16bc405173106871fc => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:00:09.404133 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3074fa32-4428-4ca5-9409-1322a6c1bfc5 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:09.443943 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea639a0-f871-4c3b-a24d-5666713e407f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:09.444728 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea639a0-f871-4c3b-a24d-5666713e407f None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:09.445017 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea639a0-f871-4c3b-a24d-5666713e407f None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:09.445300 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea639a0-f871-4c3b-a24d-5666713e407f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:09.446576 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea639a0-f871-4c3b-a24d-5666713e407f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:09.448139 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea639a0-f871-4c3b-a24d-5666713e407f None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:09.450049 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 70/140] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:00:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:09.456728 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac7d004-c8e4-42ed-ab50-167785f37650 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:09.558079 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:09.564254 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59336884-0c77-4dcc-b32a-a9fbd8d27601 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:09.564568 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59336884-0c77-4dcc-b32a-a9fbd8d27601 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:09.564813 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59336884-0c77-4dcc-b32a-a9fbd8d27601 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:09.566293 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 71/141] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:00:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:00:09.572088 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d55994-c9d9-4cd4-85b7-9103cd72dda8 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:09.572377 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d55994-c9d9-4cd4-85b7-9103cd72dda8 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:09.574661 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d55994-c9d9-4cd4-85b7-9103cd72dda8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:09.589925 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac7d004-c8e4-42ed-ab50-167785f37650 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:09.611890 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-01d55994-c9d9-4cd4-85b7-9103cd72dda8 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:00:09.613117 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-01d55994-c9d9-4cd4-85b7-9103cd72dda8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:09.652220 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a77fabd-c4de-4bc6-892d-d3516916cbc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:09.652855 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a77fabd-c4de-4bc6-892d-d3516916cbc4 None admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:09.653142 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a77fabd-c4de-4bc6-892d-d3516916cbc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:09.653423 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a77fabd-c4de-4bc6-892d-d3516916cbc4 None admin] PATH_INFO: `/v3/users` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:09.654095 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a77fabd-c4de-4bc6-892d-d3516916cbc4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:09.656200 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a77fabd-c4de-4bc6-892d-d3516916cbc4 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:09.721930 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 72/142] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:00:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 25 03:00:09.777034 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 71/143] 10.222.0.58 () {60 vars in 1132 bytes} [Thu Aug 25 03:00:09 2022] POST /identity/v3/users => generated 283 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:00:10.066392 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d55994-c9d9-4cd4-85b7-9103cd72dda8 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:10.151002 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:10.191462 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d55994-c9d9-4cd4-85b7-9103cd72dda8 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:10.231460 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0489389b-b35d-49de-8715-d38c1e267063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:10.232115 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0489389b-b35d-49de-8715-d38c1e267063 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:10.232399 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0489389b-b35d-49de-8715-d38c1e267063 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:10.232675 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0489389b-b35d-49de-8715-d38c1e267063 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:10.248737 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0489389b-b35d-49de-8715-d38c1e267063 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:00:10.254088 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-0489389b-b35d-49de-8715-d38c1e267063 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 25 03:00:10.254088 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:00:10.254088 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:00:10.254088 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:00:10.254088 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:00:10.254088 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:00:10.254088 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:00:10.254088 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:00:10.254088 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:00:10.254088 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:00:10.254088 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:00:10.254088 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:00:10.254088 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:00:10.254088 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:00:10.254088 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:00:10.254088 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:00:10.254939 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:00:10.254939 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:00:10.254939 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:00:10.254939 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:00:10.254939 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:00:10.254939 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:00:10.254939 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:00:10.254939 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:00:10.254939 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:00:10.254939 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:00:10.254939 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:00:10.254939 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:00:10.254939 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:00:10.254939 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:00:10.254939 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:00:10.254939 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:00:10.255679 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:00:10.255679 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:00:10.255679 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:00:10.255679 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:00:10.255679 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:00:10.255679 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:00:10.255679 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:00:10.255679 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:00:10.255679 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:00:10.255679 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:00:10.255679 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 25 03:00:10.255679 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:00:10.255679 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 73/144] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Aug 25 03:00:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:00:10.261127 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a77fabd-c4de-4bc6-892d-d3516916cbc4 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:10.385257 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a77fabd-c4de-4bc6-892d-d3516916cbc4 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:10.414982 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:10.420935 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec79c835-f1d8-470e-8bdb-d989a1bcfc59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:10.421556 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec79c835-f1d8-470e-8bdb-d989a1bcfc59 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:10.421845 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec79c835-f1d8-470e-8bdb-d989a1bcfc59 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:10.422118 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec79c835-f1d8-470e-8bdb-d989a1bcfc59 None admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:10.422833 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec79c835-f1d8-470e-8bdb-d989a1bcfc59 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:10.423202 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec79c835-f1d8-470e-8bdb-d989a1bcfc59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:10.424580 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec79c835-f1d8-470e-8bdb-d989a1bcfc59 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:10.433146 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 72/145] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Aug 25 03:00:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:10.439007 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0489389b-b35d-49de-8715-d38c1e267063 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:10.569891 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0489389b-b35d-49de-8715-d38c1e267063 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:10.602193 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:10.610094 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbc6a35-75db-48be-af08-18c2356abb47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:10.610787 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbc6a35-75db-48be-af08-18c2356abb47 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:10.612770 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbc6a35-75db-48be-af08-18c2356abb47 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:10.613062 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbc6a35-75db-48be-af08-18c2356abb47 None admin] PATH_INFO: `/v3/users/064052b980324793a0bc0266db01c33b` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:10.614598 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbc6a35-75db-48be-af08-18c2356abb47 None admin] RBAC: Authorizing `identity:get_user(user_id=064052b980324793a0bc0266db01c33b)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:10.616066 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbc6a35-75db-48be-af08-18c2356abb47 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:10.618051 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 74/146] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Aug 25 03:00:10 2022] GET /identity/v3/users/064052b980324793a0bc0266db01c33b => generated 289 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:10.623302 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec79c835-f1d8-470e-8bdb-d989a1bcfc59 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:10.736065 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec79c835-f1d8-470e-8bdb-d989a1bcfc59 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:10.776694 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcf0473-ca6d-4559-9f7c-d3535ade8236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:10.777339 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcf0473-ca6d-4559-9f7c-d3535ade8236 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:10.777689 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcf0473-ca6d-4559-9f7c-d3535ade8236 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:10.777974 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcf0473-ca6d-4559-9f7c-d3535ade8236 None admin] PATH_INFO: `/v3/projects/0d32389226ec4d2dac8c390a33d5436c` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:10.779205 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcf0473-ca6d-4559-9f7c-d3535ade8236 None admin] RBAC: Authorizing `identity:get_project(project_id=0d32389226ec4d2dac8c390a33d5436c)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:10.781180 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcf0473-ca6d-4559-9f7c-d3535ade8236 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:10.784228 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 73/147] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Aug 25 03:00:10 2022] GET /identity/v3/projects/0d32389226ec4d2dac8c390a33d5436c => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:10.791564 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbc6a35-75db-48be-af08-18c2356abb47 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:10.897730 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbc6a35-75db-48be-af08-18c2356abb47 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:10.934271 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ba05ed-52c9-43c3-8f78-97518171065b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:10.934987 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ba05ed-52c9-43c3-8f78-97518171065b None admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:10.935544 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ba05ed-52c9-43c3-8f78-97518171065b None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:10.935826 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ba05ed-52c9-43c3-8f78-97518171065b None admin] PATH_INFO: `/v3/projects/0d32389226ec4d2dac8c390a33d5436c/users/064052b980324793a0bc0266db01c33b/roles/3de9d2fd0b464c399f67fb5240905596` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:10.938363 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ba05ed-52c9-43c3-8f78-97518171065b None admin] RBAC: Authorizing `identity:create_grant(project_id=0d32389226ec4d2dac8c390a33d5436c, user_id=064052b980324793a0bc0266db01c33b, role_id=3de9d2fd0b464c399f67fb5240905596)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:10.939875 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ba05ed-52c9-43c3-8f78-97518171065b None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:11.029034 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 75/148] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Aug 25 03:00:10 2022] PUT /identity/v3/projects/0d32389226ec4d2dac8c390a33d5436c/users/064052b980324793a0bc0266db01c33b/roles/3de9d2fd0b464c399f67fb5240905596 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:00:11.658445 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2d6caa-f386-4b33-b718-59e8735f88f5 None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:11.658960 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2d6caa-f386-4b33-b718-59e8735f88f5 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:11.659036 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2d6caa-f386-4b33-b718-59e8735f88f5 None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:11.660367 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 74/149] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:00:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:00:11.665844 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dfa5de-7454-474f-81e6-adec5a1cd8ea None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:11.666261 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dfa5de-7454-474f-81e6-adec5a1cd8ea None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:11.666473 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dfa5de-7454-474f-81e6-adec5a1cd8ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:11.690477 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-36dfa5de-7454-474f-81e6-adec5a1cd8ea None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:00:11.691778 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-36dfa5de-7454-474f-81e6-adec5a1cd8ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:11.751131 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:11.768244 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 76/150] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:00:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 03:00:12.132753 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2d6caa-f386-4b33-b718-59e8735f88f5 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:12.209288 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:12.258984 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2d6caa-f386-4b33-b718-59e8735f88f5 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:12.296966 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cf9491-22e8-4032-acc8-6150a7024066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:12.297621 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cf9491-22e8-4032-acc8-6150a7024066 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:12.297904 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cf9491-22e8-4032-acc8-6150a7024066 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:12.298181 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cf9491-22e8-4032-acc8-6150a7024066 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:12.310021 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cf9491-22e8-4032-acc8-6150a7024066 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:00:12.315427 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-54cf9491-22e8-4032-acc8-6150a7024066 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 25 03:00:12.315427 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:00:12.315427 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:00:12.315427 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:00:12.315427 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:00:12.315427 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:00:12.315427 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:00:12.315427 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:00:12.315427 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:00:12.315427 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:00:12.315427 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:00:12.315427 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:00:12.315427 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:00:12.315427 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:00:12.315427 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:00:12.315427 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:00:12.316270 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:00:12.316270 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:00:12.316270 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:00:12.316270 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:00:12.316270 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:00:12.316270 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:00:12.316270 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:00:12.316270 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:00:12.316270 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:00:12.316270 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:00:12.316270 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:00:12.316270 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:00:12.316270 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:00:12.316270 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:00:12.316270 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:00:12.316270 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:00:12.317005 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:00:12.317005 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:00:12.317005 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:00:12.317005 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:00:12.317005 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:00:12.317005 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:00:12.317005 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:00:12.317005 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:00:12.317005 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:00:12.317005 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:00:12.317005 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 25 03:00:12.317005 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:00:12.317005 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 75/151] 10.222.0.58 () {56 vars in 1114 bytes} [Thu Aug 25 03:00:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:00:12.322763 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dfa5de-7454-474f-81e6-adec5a1cd8ea None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:12.426556 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dfa5de-7454-474f-81e6-adec5a1cd8ea None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:12.456215 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2d2b76-8404-475c-a402-5caf518344cd None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:12.456534 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2d2b76-8404-475c-a402-5caf518344cd None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:12.456773 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2d2b76-8404-475c-a402-5caf518344cd None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:12.458518 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 76/152] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:00:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:00:12.463880 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1384e1d5-8f70-4764-a4e7-f909a76b5502 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:12.464179 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1384e1d5-8f70-4764-a4e7-f909a76b5502 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:12.464436 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1384e1d5-8f70-4764-a4e7-f909a76b5502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:12.484273 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb35e2cf-414a-4e67-b41f-c67c06a26bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:12.484916 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb35e2cf-414a-4e67-b41f-c67c06a26bfc None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:12.485193 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb35e2cf-414a-4e67-b41f-c67c06a26bfc None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:12.485463 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb35e2cf-414a-4e67-b41f-c67c06a26bfc None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:12.486163 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb35e2cf-414a-4e67-b41f-c67c06a26bfc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:12.486499 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb35e2cf-414a-4e67-b41f-c67c06a26bfc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:12.487939 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb35e2cf-414a-4e67-b41f-c67c06a26bfc None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:12.495158 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-1384e1d5-8f70-4764-a4e7-f909a76b5502 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:00:12.496377 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-1384e1d5-8f70-4764-a4e7-f909a76b5502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:12.514617 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 77/153] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Aug 25 03:00:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:12.518541 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb35e2cf-414a-4e67-b41f-c67c06a26bfc None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:12.618643 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:12.642366 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 77/154] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:00:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 25 03:00:12.677859 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb35e2cf-414a-4e67-b41f-c67c06a26bfc None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:12.712562 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571ec3ec-7659-4f78-bedc-5aca5eb79871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:12.713218 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571ec3ec-7659-4f78-bedc-5aca5eb79871 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:12.713506 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571ec3ec-7659-4f78-bedc-5aca5eb79871 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:12.713790 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571ec3ec-7659-4f78-bedc-5aca5eb79871 None admin] PATH_INFO: `/v3/users/26e7c9fc5d3c41de8156071d9f0cb3cd` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:12.714964 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571ec3ec-7659-4f78-bedc-5aca5eb79871 None admin] RBAC: Authorizing `identity:get_user(user_id=26e7c9fc5d3c41de8156071d9f0cb3cd)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:12.716650 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571ec3ec-7659-4f78-bedc-5aca5eb79871 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:12.718843 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 78/155] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Aug 25 03:00:12 2022] GET /identity/v3/users/26e7c9fc5d3c41de8156071d9f0cb3cd => generated 275 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:00:12.724583 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1384e1d5-8f70-4764-a4e7-f909a76b5502 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:12.799228 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:12.872224 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1384e1d5-8f70-4764-a4e7-f909a76b5502 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:12.916578 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f64b1e-bf5f-49b5-93fa-3977ef31a604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:12.917202 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f64b1e-bf5f-49b5-93fa-3977ef31a604 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:12.917483 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f64b1e-bf5f-49b5-93fa-3977ef31a604 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:12.917754 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f64b1e-bf5f-49b5-93fa-3977ef31a604 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:12.978122 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143f6354-d9c8-41b9-90a3-67b528597027 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:12.978858 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143f6354-d9c8-41b9-90a3-67b528597027 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:12.979358 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143f6354-d9c8-41b9-90a3-67b528597027 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:12.981143 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 79/156] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:00:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 25 03:00:12.989645 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f87820-424f-40da-9d6e-c18eef0a27d6 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:12.990235 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f87820-424f-40da-9d6e-c18eef0a27d6 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:12.990918 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f87820-424f-40da-9d6e-c18eef0a27d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:13.023841 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-a9f87820-424f-40da-9d6e-c18eef0a27d6 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:00:13.025281 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-a9f87820-424f-40da-9d6e-c18eef0a27d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:13.026225 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f64b1e-bf5f-49b5-93fa-3977ef31a604 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:13.027977 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f64b1e-bf5f-49b5-93fa-3977ef31a604 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:13.171680 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 80/157] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:00:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 03:00:13.173291 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:13.174371 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcc2daf-f284-4419-80ea-7a5ed8b0d92d None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:13.174657 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcc2daf-f284-4419-80ea-7a5ed8b0d92d None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:13.174927 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcc2daf-f284-4419-80ea-7a5ed8b0d92d None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:13.176196 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 81/158] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:00:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:00:13.176890 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcc2daf-f284-4419-80ea-7a5ed8b0d92d None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:13.296792 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 78/159] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Aug 25 03:00:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 573 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 25 03:00:13.298062 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0995970d-647f-4eae-8035-9f30eebc77f6 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:13.298504 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0995970d-647f-4eae-8035-9f30eebc77f6 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:13.298786 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0995970d-647f-4eae-8035-9f30eebc77f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:13.310981 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:13.346988 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-0995970d-647f-4eae-8035-9f30eebc77f6 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:00:13.348241 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-0995970d-647f-4eae-8035-9f30eebc77f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:13.432271 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcc2daf-f284-4419-80ea-7a5ed8b0d92d None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:13.503907 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de769b42-92c6-47b5-b808-13c788a28369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:13.504622 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de769b42-92c6-47b5-b808-13c788a28369 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:13.505061 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 79/160] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:00:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 03:00:13.505177 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de769b42-92c6-47b5-b808-13c788a28369 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:13.505177 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de769b42-92c6-47b5-b808-13c788a28369 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:13.505723 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0995970d-647f-4eae-8035-9f30eebc77f6 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:13.531515 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de769b42-92c6-47b5-b808-13c788a28369 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:00:13.536302 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-de769b42-92c6-47b5-b808-13c788a28369 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 03:00:13.536302 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:00:13.536302 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:00:13.536302 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:00:13.536302 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:00:13.536302 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:00:13.536302 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:00:13.536302 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:00:13.536302 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:00:13.536302 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:00:13.536302 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:00:13.536302 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:00:13.536302 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:00:13.536302 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:00:13.536302 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:00:13.536302 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:00:13.537204 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:00:13.537204 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:00:13.537204 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:00:13.537204 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:00:13.537204 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:00:13.537204 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:00:13.537204 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:00:13.537204 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:00:13.537204 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:00:13.537204 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:00:13.537204 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:00:13.537204 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:00:13.537204 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:00:13.537204 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:00:13.537204 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:00:13.537204 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:00:13.538048 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:00:13.538048 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:00:13.538048 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:00:13.538048 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:00:13.538048 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:00:13.538048 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:00:13.538048 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:00:13.538048 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:00:13.538048 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:00:13.538048 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:00:13.538048 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 03:00:13.538048 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:00:13.538048 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 82/161] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Aug 25 03:00:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 361 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:00:13.544603 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de769b42-92c6-47b5-b808-13c788a28369 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:13.648260 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:13.667869 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0995970d-647f-4eae-8035-9f30eebc77f6 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:13.707884 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7791c98-b62a-45d0-acdb-f3344f707af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:13.708799 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7791c98-b62a-45d0-acdb-f3344f707af9 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:13.709236 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7791c98-b62a-45d0-acdb-f3344f707af9 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:13.709662 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7791c98-b62a-45d0-acdb-f3344f707af9 None admin] PATH_INFO: `/v3/projects/0d32389226ec4d2dac8c390a33d5436c` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:13.710975 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7791c98-b62a-45d0-acdb-f3344f707af9 None admin] RBAC: Authorizing `identity:get_project(project_id=0d32389226ec4d2dac8c390a33d5436c)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:13.712617 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7791c98-b62a-45d0-acdb-f3344f707af9 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:13.714598 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 80/162] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Aug 25 03:00:13 2022] GET /identity/v3/projects/0d32389226ec4d2dac8c390a33d5436c => generated 298 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:13.725712 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7791c98-b62a-45d0-acdb-f3344f707af9 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:13.765742 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de769b42-92c6-47b5-b808-13c788a28369 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:13.781899 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:13.850067 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9144ed64-7554-45f6-b77b-44236d870cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:13.850771 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9144ed64-7554-45f6-b77b-44236d870cd1 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:13.851057 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9144ed64-7554-45f6-b77b-44236d870cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:13.851330 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9144ed64-7554-45f6-b77b-44236d870cd1 None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:13.852012 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9144ed64-7554-45f6-b77b-44236d870cd1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:13.852316 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9144ed64-7554-45f6-b77b-44236d870cd1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:13.853711 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9144ed64-7554-45f6-b77b-44236d870cd1 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:13.868030 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 83/163] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Aug 25 03:00:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:13.869052 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9144ed64-7554-45f6-b77b-44236d870cd1 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:13.870469 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7791c98-b62a-45d0-acdb-f3344f707af9 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:13.921692 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0ceaf9-63f5-471d-97d0-99827d5bd4d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:13.922325 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0ceaf9-63f5-471d-97d0-99827d5bd4d6 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:13.922613 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0ceaf9-63f5-471d-97d0-99827d5bd4d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:13.922886 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0ceaf9-63f5-471d-97d0-99827d5bd4d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:13.923630 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0ceaf9-63f5-471d-97d0-99827d5bd4d6 None admin] RBAC: Adding query filter params (role.id=7886cc7e333a42d08085932494a3ea6b, scope.project.id=0d32389226ec4d2dac8c390a33d5436c, user.id=26e7c9fc5d3c41de8156071d9f0cb3cd) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:13.923932 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0ceaf9-63f5-471d-97d0-99827d5bd4d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:13.925398 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0ceaf9-63f5-471d-97d0-99827d5bd4d6 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:13.935652 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 81/164] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Aug 25 03:00:13 2022] GET /identity/v3/role_assignments?user.id=26e7c9fc5d3c41de8156071d9f0cb3cd&scope.project.id=0d32389226ec4d2dac8c390a33d5436c&role.id=7886cc7e333a42d08085932494a3ea6b => generated 608 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:00:13.936857 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0ceaf9-63f5-471d-97d0-99827d5bd4d6 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:13.978565 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:14.023833 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9144ed64-7554-45f6-b77b-44236d870cd1 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:14.075965 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f85def-3bf9-4f07-a247-530182bc3029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:14.077663 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f85def-3bf9-4f07-a247-530182bc3029 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:14.077955 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f85def-3bf9-4f07-a247-530182bc3029 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:14.078229 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f85def-3bf9-4f07-a247-530182bc3029 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:14.100300 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f85def-3bf9-4f07-a247-530182bc3029 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:00:14.105648 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-70f85def-3bf9-4f07-a247-530182bc3029 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 25 03:00:14.105648 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:00:14.105648 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:00:14.105648 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:00:14.105648 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:00:14.105648 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:00:14.105648 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:00:14.105648 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:00:14.105648 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:00:14.105648 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:00:14.105648 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:00:14.105648 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:00:14.105648 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:00:14.105648 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:00:14.105648 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:00:14.105648 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:00:14.106578 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:00:14.106578 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:00:14.106578 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:00:14.106578 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:00:14.106578 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:00:14.106578 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:00:14.106578 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:00:14.106578 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:00:14.106578 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:00:14.106578 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:00:14.106578 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:00:14.106578 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:00:14.106578 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:00:14.106578 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:00:14.106578 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:00:14.106578 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:00:14.107437 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:00:14.107437 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:00:14.107437 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:00:14.107437 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:00:14.107437 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:00:14.107437 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:00:14.107437 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:00:14.107437 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:00:14.107437 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:00:14.107437 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:00:14.107437 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 25 03:00:14.107437 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:00:14.107437 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 84/165] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Aug 25 03:00:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:00:14.108782 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8338aee-93c3-446c-a2c3-2f15a3b03793 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:14.109657 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8338aee-93c3-446c-a2c3-2f15a3b03793 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:14.113897 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8338aee-93c3-446c-a2c3-2f15a3b03793 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:14.115328 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 85/166] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:00:14 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:00:14.117440 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8338aee-93c3-446c-a2c3-2f15a3b03793 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:14.125943 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0ceaf9-63f5-471d-97d0-99827d5bd4d6 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:14.192257 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3884366-79ae-40e8-bfc3-66bdd4444fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:14.192847 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3884366-79ae-40e8-bfc3-66bdd4444fc6 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:14.193129 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3884366-79ae-40e8-bfc3-66bdd4444fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:14.193404 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3884366-79ae-40e8-bfc3-66bdd4444fc6 None admin] PATH_INFO: `/v3/users/e8b708e7077c461bb73c230d7b1825fb` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:14.194642 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3884366-79ae-40e8-bfc3-66bdd4444fc6 None admin] RBAC: Authorizing `identity:get_user(user_id=e8b708e7077c461bb73c230d7b1825fb)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:14.196162 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3884366-79ae-40e8-bfc3-66bdd4444fc6 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:14.198072 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 82/167] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Aug 25 03:00:13 2022] GET /identity/v3/users/e8b708e7077c461bb73c230d7b1825fb => generated 247 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:14.199149 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b84de2-ad1c-4dde-8b53-c7ec31337f07 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:14.199430 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b84de2-ad1c-4dde-8b53-c7ec31337f07 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:14.199706 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b84de2-ad1c-4dde-8b53-c7ec31337f07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:14.210776 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:14.232341 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-12b84de2-ad1c-4dde-8b53-c7ec31337f07 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:00:14.234168 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-12b84de2-ad1c-4dde-8b53-c7ec31337f07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:14.340960 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 83/168] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:00:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 03:00:14.341826 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b84de2-ad1c-4dde-8b53-c7ec31337f07 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:14.382659 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:14.394990 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8338aee-93c3-446c-a2c3-2f15a3b03793 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:14.487217 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0544a862-55a2-4e30-893f-1303daca18aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:14.487895 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0544a862-55a2-4e30-893f-1303daca18aa None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:14.488399 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0544a862-55a2-4e30-893f-1303daca18aa None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:14.488883 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0544a862-55a2-4e30-893f-1303daca18aa None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:14.490101 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0544a862-55a2-4e30-893f-1303daca18aa None admin] RBAC: Adding query filter params (name=member) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:14.490740 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0544a862-55a2-4e30-893f-1303daca18aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:14.493312 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0544a862-55a2-4e30-893f-1303daca18aa None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:14.497103 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b84de2-ad1c-4dde-8b53-c7ec31337f07 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:14.512601 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 86/169] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Aug 25 03:00:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 395 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:14.513483 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0544a862-55a2-4e30-893f-1303daca18aa None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:14.553107 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd3ea9c-5c69-4587-b885-4d36ec40e5fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:14.553734 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd3ea9c-5c69-4587-b885-4d36ec40e5fd None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:14.554011 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd3ea9c-5c69-4587-b885-4d36ec40e5fd None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:14.554302 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd3ea9c-5c69-4587-b885-4d36ec40e5fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:14.643048 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd3ea9c-5c69-4587-b885-4d36ec40e5fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:14.645292 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd3ea9c-5c69-4587-b885-4d36ec40e5fd None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:14.659614 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:14.745502 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:14.760692 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0544a862-55a2-4e30-893f-1303daca18aa None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:14.791068 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 84/170] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Aug 25 03:00:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 25 03:00:14.792686 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd3ea9c-5c69-4587-b885-4d36ec40e5fd None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:14.817673 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ae936e-f355-42d3-b1a4-f167da099e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:14.818355 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ae936e-f355-42d3-b1a4-f167da099e8e None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:14.818620 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ae936e-f355-42d3-b1a4-f167da099e8e None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:14.818895 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ae936e-f355-42d3-b1a4-f167da099e8e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:14.833603 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ae936e-f355-42d3-b1a4-f167da099e8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:00:14.838455 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-98ae936e-f355-42d3-b1a4-f167da099e8e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 25 03:00:14.838455 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:00:14.838455 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:00:14.838455 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:00:14.838455 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:00:14.838455 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:00:14.838455 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:00:14.838455 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:00:14.838455 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:00:14.838455 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:00:14.838455 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:00:14.838455 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:00:14.838455 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:00:14.838455 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:00:14.838455 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:00:14.838455 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:00:14.839449 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:00:14.839449 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:00:14.839449 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:00:14.839449 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:00:14.839449 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:00:14.839449 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:00:14.839449 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:00:14.839449 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:00:14.839449 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:00:14.839449 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:00:14.839449 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:00:14.839449 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:00:14.839449 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:00:14.839449 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:00:14.839449 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:00:14.839449 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:00:14.840217 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:00:14.840217 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:00:14.840217 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:00:14.840217 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:00:14.840217 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:00:14.840217 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:00:14.840217 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:00:14.840217 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:00:14.840217 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:00:14.840217 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:00:14.840217 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 25 03:00:14.840217 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:00:14.840217 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 87/171] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Aug 25 03:00:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:00:14.846330 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2bf31c-1137-4a63-9d85-2e9379c72847 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:14.846330 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2bf31c-1137-4a63-9d85-2e9379c72847 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:14.846330 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2bf31c-1137-4a63-9d85-2e9379c72847 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:14.846330 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 88/172] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:00:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:00:14.846330 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2bf31c-1137-4a63-9d85-2e9379c72847 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:14.947746 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd3ea9c-5c69-4587-b885-4d36ec40e5fd None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:14.960298 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2bf31c-1137-4a63-9d85-2e9379c72847 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:14.964709 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:14.984738 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23b4483-c56c-49b4-8df6-2b68942df28d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:14.985623 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23b4483-c56c-49b4-8df6-2b68942df28d None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:14.986141 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23b4483-c56c-49b4-8df6-2b68942df28d None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:14.986626 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23b4483-c56c-49b4-8df6-2b68942df28d None admin] PATH_INFO: `/v3/users/26e7c9fc5d3c41de8156071d9f0cb3cd` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:14.987765 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23b4483-c56c-49b4-8df6-2b68942df28d None admin] RBAC: Authorizing `identity:get_user(user_id=26e7c9fc5d3c41de8156071d9f0cb3cd)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:14.989212 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23b4483-c56c-49b4-8df6-2b68942df28d None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:14.995398 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 85/173] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Aug 25 03:00:14 2022] GET /identity/v3/users/26e7c9fc5d3c41de8156071d9f0cb3cd => generated 275 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:14.995398 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23b4483-c56c-49b4-8df6-2b68942df28d None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:15.004898 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab35501-bd54-4fb1-9bcc-a7d222c12af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:15.004898 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab35501-bd54-4fb1-9bcc-a7d222c12af1 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:15.004898 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab35501-bd54-4fb1-9bcc-a7d222c12af1 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:15.004898 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab35501-bd54-4fb1-9bcc-a7d222c12af1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:15.016783 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab35501-bd54-4fb1-9bcc-a7d222c12af1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:00:15.024050 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-2ab35501-bd54-4fb1-9bcc-a7d222c12af1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 03:00:15.024050 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:00:15.024050 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:00:15.024050 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:00:15.024050 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:00:15.024050 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:00:15.024050 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:00:15.024050 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:00:15.024050 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:00:15.024050 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:00:15.024050 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:00:15.024050 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:00:15.024050 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:00:15.024050 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:00:15.024050 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:00:15.024050 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:00:15.024980 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:00:15.024980 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:00:15.024980 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:00:15.024980 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:00:15.024980 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:00:15.024980 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:00:15.024980 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:00:15.024980 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:00:15.024980 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:00:15.024980 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:00:15.024980 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:00:15.024980 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:00:15.024980 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:00:15.024980 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:00:15.024980 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:00:15.024980 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:00:15.025736 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:00:15.025736 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:00:15.025736 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:00:15.025736 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:00:15.025736 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:00:15.025736 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:00:15.025736 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:00:15.025736 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:00:15.025736 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:00:15.025736 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:00:15.025736 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 03:00:15.025736 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:00:15.025736 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 89/174] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Aug 25 03:00:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:00:15.029744 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab35501-bd54-4fb1-9bcc-a7d222c12af1 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:15.043452 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:15.101600 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23b4483-c56c-49b4-8df6-2b68942df28d None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:15.137835 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab35501-bd54-4fb1-9bcc-a7d222c12af1 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:15.139844 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389c3169-f195-4d09-b765-2251e74f8db8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:15.140694 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389c3169-f195-4d09-b765-2251e74f8db8 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:15.140979 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389c3169-f195-4d09-b765-2251e74f8db8 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:15.141261 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389c3169-f195-4d09-b765-2251e74f8db8 None admin] PATH_INFO: `/v3/projects/0d32389226ec4d2dac8c390a33d5436c` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:15.142423 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389c3169-f195-4d09-b765-2251e74f8db8 None admin] RBAC: Authorizing `identity:get_project(project_id=0d32389226ec4d2dac8c390a33d5436c)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:15.143883 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389c3169-f195-4d09-b765-2251e74f8db8 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:15.150497 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 86/175] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Aug 25 03:00:14 2022] GET /identity/v3/projects/0d32389226ec4d2dac8c390a33d5436c => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:15.151720 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2798d16-a1d6-43ca-aef0-861fd33ea690 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:15.152004 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2798d16-a1d6-43ca-aef0-861fd33ea690 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:15.152288 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2798d16-a1d6-43ca-aef0-861fd33ea690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:15.171151 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-d2798d16-a1d6-43ca-aef0-861fd33ea690 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:00:15.172341 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-d2798d16-a1d6-43ca-aef0-861fd33ea690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:15.176149 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4112e9-076f-44db-8adb-b3f9abe12d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:15.176760 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4112e9-076f-44db-8adb-b3f9abe12d93 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:15.177035 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4112e9-076f-44db-8adb-b3f9abe12d93 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:15.177303 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4112e9-076f-44db-8adb-b3f9abe12d93 None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:15.177964 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4112e9-076f-44db-8adb-b3f9abe12d93 None admin] RBAC: Adding query filter params (name=member) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:15.178260 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4112e9-076f-44db-8adb-b3f9abe12d93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:15.179650 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4112e9-076f-44db-8adb-b3f9abe12d93 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:15.193050 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 90/176] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Aug 25 03:00:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:15.193050 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344e27af-1c2d-4c57-a235-740bfc6eba40 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:15.193050 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344e27af-1c2d-4c57-a235-740bfc6eba40 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:15.193050 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344e27af-1c2d-4c57-a235-740bfc6eba40 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:15.193050 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 91/177] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:00:15 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:00:15.193844 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344e27af-1c2d-4c57-a235-740bfc6eba40 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:15.226029 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:15.252829 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 87/178] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:00:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 03:00:15.253899 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2798d16-a1d6-43ca-aef0-861fd33ea690 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:15.272211 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:15.304102 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344e27af-1c2d-4c57-a235-740bfc6eba40 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:15.341556 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b0d879-1d67-43f6-ac48-abc587326bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:15.342172 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b0d879-1d67-43f6-ac48-abc587326bf3 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:15.343031 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b0d879-1d67-43f6-ac48-abc587326bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:15.343329 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b0d879-1d67-43f6-ac48-abc587326bf3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:15.359664 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2798d16-a1d6-43ca-aef0-861fd33ea690 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:15.395069 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd9a9d8-ffc9-446e-8b0f-c7e631d38b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:15.396235 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd9a9d8-ffc9-446e-8b0f-c7e631d38b0b None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:15.396619 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd9a9d8-ffc9-446e-8b0f-c7e631d38b0b None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:15.397042 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd9a9d8-ffc9-446e-8b0f-c7e631d38b0b None admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:15.397765 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd9a9d8-ffc9-446e-8b0f-c7e631d38b0b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:15.398174 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd9a9d8-ffc9-446e-8b0f-c7e631d38b0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:15.399677 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd9a9d8-ffc9-446e-8b0f-c7e631d38b0b None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:15.413691 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 88/179] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Aug 25 03:00:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:15.413691 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd9a9d8-ffc9-446e-8b0f-c7e631d38b0b None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:15.413930 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b0d879-1d67-43f6-ac48-abc587326bf3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:15.413930 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b0d879-1d67-43f6-ac48-abc587326bf3 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:15.458840 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:15.484138 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:15.514849 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 92/180] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Aug 25 03:00:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 25 03:00:15.516178 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0dc7d1-0a40-4c6f-b4d4-f66038e4fbc8 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:15.516466 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0dc7d1-0a40-4c6f-b4d4-f66038e4fbc8 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:15.516743 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0dc7d1-0a40-4c6f-b4d4-f66038e4fbc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:15.524897 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd9a9d8-ffc9-446e-8b0f-c7e631d38b0b None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:15.535156 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-db0dc7d1-0a40-4c6f-b4d4-f66038e4fbc8 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:00:15.536363 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-db0dc7d1-0a40-4c6f-b4d4-f66038e4fbc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:15.558718 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d765dc31-fb2f-46fa-b024-f8407f9e4e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:15.559331 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d765dc31-fb2f-46fa-b024-f8407f9e4e59 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:15.559610 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d765dc31-fb2f-46fa-b024-f8407f9e4e59 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:15.559887 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d765dc31-fb2f-46fa-b024-f8407f9e4e59 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:15.560613 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d765dc31-fb2f-46fa-b024-f8407f9e4e59 None admin] RBAC: Adding query filter params (role.id=62f19ea272b447ccaab1576cd9bf4fd9, scope.project.id=0d32389226ec4d2dac8c390a33d5436c, user.id=e8b708e7077c461bb73c230d7b1825fb) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:15.560918 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d765dc31-fb2f-46fa-b024-f8407f9e4e59 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:15.562971 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d765dc31-fb2f-46fa-b024-f8407f9e4e59 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:15.569043 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 89/181] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Aug 25 03:00:15 2022] GET /identity/v3/role_assignments?user.id=e8b708e7077c461bb73c230d7b1825fb&scope.project.id=0d32389226ec4d2dac8c390a33d5436c&role.id=62f19ea272b447ccaab1576cd9bf4fd9 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:15.569926 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d765dc31-fb2f-46fa-b024-f8407f9e4e59 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:15.609066 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 93/182] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:00:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 03:00:15.610271 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951c960d-d04a-4d7c-bc28-cfff8afab902 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:15.610615 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951c960d-d04a-4d7c-bc28-cfff8afab902 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:15.610894 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951c960d-d04a-4d7c-bc28-cfff8afab902 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:15.612349 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 94/183] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:00:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:00:15.612888 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951c960d-d04a-4d7c-bc28-cfff8afab902 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:15.716560 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d765dc31-fb2f-46fa-b024-f8407f9e4e59 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:15.718556 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:15.753007 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951c960d-d04a-4d7c-bc28-cfff8afab902 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:15.754634 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:15.763005 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faee075-2db2-488a-bc82-8730f0797b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:15.769124 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faee075-2db2-488a-bc82-8730f0797b2f None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:15.769124 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faee075-2db2-488a-bc82-8730f0797b2f None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:15.769124 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faee075-2db2-488a-bc82-8730f0797b2f None admin] PATH_INFO: `/v3/users/26e7c9fc5d3c41de8156071d9f0cb3cd` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:15.769124 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faee075-2db2-488a-bc82-8730f0797b2f None admin] RBAC: Authorizing `identity:get_user(user_id=26e7c9fc5d3c41de8156071d9f0cb3cd)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:15.769124 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faee075-2db2-488a-bc82-8730f0797b2f None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:15.769766 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 90/184] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Aug 25 03:00:15 2022] GET /identity/v3/users/26e7c9fc5d3c41de8156071d9f0cb3cd => generated 275 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:15.775375 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faee075-2db2-488a-bc82-8730f0797b2f None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:15.796865 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40517247-f399-4f8b-89d2-d66e0ab81197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:15.797613 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40517247-f399-4f8b-89d2-d66e0ab81197 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:15.797895 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40517247-f399-4f8b-89d2-d66e0ab81197 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:15.798184 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40517247-f399-4f8b-89d2-d66e0ab81197 None admin] PATH_INFO: `/v3/users/90cfb8b65288490aadfab2795371d6e2` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:15.813774 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40517247-f399-4f8b-89d2-d66e0ab81197 None admin] RBAC: Authorizing `identity:get_user(user_id=90cfb8b65288490aadfab2795371d6e2)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:15.815290 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40517247-f399-4f8b-89d2-d66e0ab81197 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:15.819214 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 95/185] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Aug 25 03:00:15 2022] GET /identity/v3/users/90cfb8b65288490aadfab2795371d6e2 => generated 283 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:15.819214 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40517247-f399-4f8b-89d2-d66e0ab81197 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:15.891368 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faee075-2db2-488a-bc82-8730f0797b2f None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:15.926615 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40517247-f399-4f8b-89d2-d66e0ab81197 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:15.927614 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5270d33-451f-4249-90f8-69974da2d142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:15.928249 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5270d33-451f-4249-90f8-69974da2d142 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:15.928528 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5270d33-451f-4249-90f8-69974da2d142 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:15.928805 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5270d33-451f-4249-90f8-69974da2d142 None admin] PATH_INFO: `/v3/projects/0d32389226ec4d2dac8c390a33d5436c` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:15.929928 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5270d33-451f-4249-90f8-69974da2d142 None admin] RBAC: Authorizing `identity:get_project(project_id=0d32389226ec4d2dac8c390a33d5436c)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:15.931420 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5270d33-451f-4249-90f8-69974da2d142 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:15.933241 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 91/186] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Aug 25 03:00:15 2022] GET /identity/v3/projects/0d32389226ec4d2dac8c390a33d5436c => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:15.934161 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a180260-7299-4061-96f8-c545dbcfd364 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:15.934468 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a180260-7299-4061-96f8-c545dbcfd364 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:15.934743 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a180260-7299-4061-96f8-c545dbcfd364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:15.953361 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-5a180260-7299-4061-96f8-c545dbcfd364 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:00:15.957943 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-5a180260-7299-4061-96f8-c545dbcfd364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:15.965048 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0b103a-c65b-4229-afc9-b0fd8847722d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:15.965691 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0b103a-c65b-4229-afc9-b0fd8847722d None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:15.965974 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0b103a-c65b-4229-afc9-b0fd8847722d None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:15.966249 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0b103a-c65b-4229-afc9-b0fd8847722d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:15.976335 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0b103a-c65b-4229-afc9-b0fd8847722d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:00:15.981042 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-9b0b103a-c65b-4229-afc9-b0fd8847722d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 25 03:00:15.981042 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:00:15.981042 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:00:15.981042 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:00:15.981042 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:00:15.981042 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:00:15.981042 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:00:15.981042 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:00:15.981042 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:00:15.981042 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:00:15.981042 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:00:15.981042 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:00:15.981042 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:00:15.981042 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:00:15.981042 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:00:15.981042 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:00:15.981923 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:00:15.981923 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:00:15.981923 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:00:15.981923 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:00:15.981923 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:00:15.981923 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:00:15.981923 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:00:15.981923 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:00:15.981923 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:00:15.981923 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:00:15.981923 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:00:15.981923 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:00:15.981923 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:00:15.981923 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:00:15.981923 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:00:15.981923 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:00:15.983652 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:00:15.983652 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:00:15.983652 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:00:15.983652 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:00:15.983652 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:00:15.983652 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:00:15.983652 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:00:15.983652 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:00:15.983652 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:00:15.983652 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:00:15.983652 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 25 03:00:15.983652 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:00:15.983652 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 96/187] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Aug 25 03:00:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:00:15.983652 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0b103a-c65b-4229-afc9-b0fd8847722d None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:15.987224 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:16.006477 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:16.023516 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 92/188] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:00:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 03:00:16.024339 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a180260-7299-4061-96f8-c545dbcfd364 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:16.087476 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0b103a-c65b-4229-afc9-b0fd8847722d None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:16.122907 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04451b42-4875-4d39-9d02-a70c207cc45d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:16.123524 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04451b42-4875-4d39-9d02-a70c207cc45d None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:16.123937 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04451b42-4875-4d39-9d02-a70c207cc45d None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:16.124734 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04451b42-4875-4d39-9d02-a70c207cc45d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:16.149617 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a180260-7299-4061-96f8-c545dbcfd364 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:16.187742 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d70991-c1d5-4cd0-94c1-a927f1a220ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:16.188349 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d70991-c1d5-4cd0-94c1-a927f1a220ad None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:16.188626 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d70991-c1d5-4cd0-94c1-a927f1a220ad None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:16.188900 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d70991-c1d5-4cd0-94c1-a927f1a220ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:16.192780 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04451b42-4875-4d39-9d02-a70c207cc45d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:16.194250 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04451b42-4875-4d39-9d02-a70c207cc45d None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:16.216780 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:16.225466 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:16.258637 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d70991-c1d5-4cd0-94c1-a927f1a220ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:16.260254 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d70991-c1d5-4cd0-94c1-a927f1a220ad None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:16.298158 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 97/189] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Aug 25 03:00:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 25 03:00:16.299215 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04451b42-4875-4d39-9d02-a70c207cc45d None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:16.361947 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 93/190] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Aug 25 03:00:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 25 03:00:16.363216 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d70991-c1d5-4cd0-94c1-a927f1a220ad None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:16.399694 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04451b42-4875-4d39-9d02-a70c207cc45d None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:16.419271 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:16.427352 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:16.432641 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcac9b71-fea3-4241-8e9d-5474c73bfcbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:16.433271 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcac9b71-fea3-4241-8e9d-5474c73bfcbf None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:16.433555 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcac9b71-fea3-4241-8e9d-5474c73bfcbf None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:16.433831 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcac9b71-fea3-4241-8e9d-5474c73bfcbf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:16.444796 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcac9b71-fea3-4241-8e9d-5474c73bfcbf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:00:16.449497 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-fcac9b71-fea3-4241-8e9d-5474c73bfcbf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 25 03:00:16.449497 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:00:16.449497 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:00:16.449497 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:00:16.449497 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:00:16.449497 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:00:16.449497 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:00:16.449497 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:00:16.449497 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:00:16.449497 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:00:16.449497 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:00:16.449497 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:00:16.449497 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:00:16.449497 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:00:16.449497 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:00:16.449497 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:00:16.450471 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:00:16.450471 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:00:16.450471 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:00:16.450471 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:00:16.450471 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:00:16.450471 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:00:16.450471 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:00:16.450471 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:00:16.450471 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:00:16.450471 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:00:16.450471 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:00:16.450471 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:00:16.450471 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:00:16.450471 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:00:16.450471 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:00:16.450471 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:00:16.451448 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:00:16.451448 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:00:16.451448 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:00:16.451448 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:00:16.451448 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:00:16.451448 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:00:16.451448 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:00:16.451448 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:00:16.451448 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:00:16.451448 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:00:16.451448 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 25 03:00:16.451448 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:00:16.451448 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 98/191] 10.222.0.58 () {56 vars in 1114 bytes} [Thu Aug 25 03:00:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:00:16.451448 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcac9b71-fea3-4241-8e9d-5474c73bfcbf None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:16.460184 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d70991-c1d5-4cd0-94c1-a927f1a220ad None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:16.492750 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118baffc-83a1-411b-86a4-f7a7d7d00938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:16.493358 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118baffc-83a1-411b-86a4-f7a7d7d00938 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:16.493638 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118baffc-83a1-411b-86a4-f7a7d7d00938 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:16.493922 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118baffc-83a1-411b-86a4-f7a7d7d00938 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:16.494678 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118baffc-83a1-411b-86a4-f7a7d7d00938 None admin] RBAC: Adding query filter params (role.id=3255f77192524a16bc405173106871fc, scope.project.id=0d32389226ec4d2dac8c390a33d5436c, user.id=26e7c9fc5d3c41de8156071d9f0cb3cd) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:16.494986 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118baffc-83a1-411b-86a4-f7a7d7d00938 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:16.496512 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118baffc-83a1-411b-86a4-f7a7d7d00938 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:16.502729 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 94/192] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Aug 25 03:00:16 2022] GET /identity/v3/role_assignments?user.id=26e7c9fc5d3c41de8156071d9f0cb3cd&scope.project.id=0d32389226ec4d2dac8c390a33d5436c&role.id=3255f77192524a16bc405173106871fc => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:16.503524 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118baffc-83a1-411b-86a4-f7a7d7d00938 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:16.571964 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcac9b71-fea3-4241-8e9d-5474c73bfcbf None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:16.616831 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd794b5-14cf-4b9c-810c-02fafa2e0408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:16.617480 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd794b5-14cf-4b9c-810c-02fafa2e0408 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:16.618055 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd794b5-14cf-4b9c-810c-02fafa2e0408 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:16.618444 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd794b5-14cf-4b9c-810c-02fafa2e0408 None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:16.619968 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd794b5-14cf-4b9c-810c-02fafa2e0408 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:16.620356 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd794b5-14cf-4b9c-810c-02fafa2e0408 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:16.621689 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118baffc-83a1-411b-86a4-f7a7d7d00938 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:16.621988 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd794b5-14cf-4b9c-810c-02fafa2e0408 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:16.633240 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 99/193] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Aug 25 03:00:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:16.633976 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd794b5-14cf-4b9c-810c-02fafa2e0408 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:16.661615 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:16.668206 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3e5368-f904-4fe2-bce3-a1384efc2901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:16.668981 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3e5368-f904-4fe2-bce3-a1384efc2901 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:16.669263 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3e5368-f904-4fe2-bce3-a1384efc2901 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:16.679187 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3e5368-f904-4fe2-bce3-a1384efc2901 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:16.679187 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3e5368-f904-4fe2-bce3-a1384efc2901 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:16.679187 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3e5368-f904-4fe2-bce3-a1384efc2901 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:16.679187 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 95/194] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:00:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:16.679187 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3e5368-f904-4fe2-bce3-a1384efc2901 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:16.688291 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:16.750363 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd794b5-14cf-4b9c-810c-02fafa2e0408 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:16.783234 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3e5368-f904-4fe2-bce3-a1384efc2901 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:16.783703 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a80e5a4-9264-4b2d-a738-f8b61d99a5c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:16.784335 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a80e5a4-9264-4b2d-a738-f8b61d99a5c0 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:16.784624 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a80e5a4-9264-4b2d-a738-f8b61d99a5c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:16.784897 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a80e5a4-9264-4b2d-a738-f8b61d99a5c0 None admin] PATH_INFO: `/v3/projects/4a3808105deb490daa12dd1f297e4b06` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:16.786005 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a80e5a4-9264-4b2d-a738-f8b61d99a5c0 None admin] RBAC: Authorizing `identity:get_project(project_id=4a3808105deb490daa12dd1f297e4b06)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:16.787498 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a80e5a4-9264-4b2d-a738-f8b61d99a5c0 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:16.789122 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 100/195] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Aug 25 03:00:16 2022] GET /identity/v3/projects/4a3808105deb490daa12dd1f297e4b06 => generated 312 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:16.794155 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a80e5a4-9264-4b2d-a738-f8b61d99a5c0 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:16.817886 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d3e9cc-03da-4d41-b13a-8ff1f1040f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:16.818545 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d3e9cc-03da-4d41-b13a-8ff1f1040f29 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:16.818829 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d3e9cc-03da-4d41-b13a-8ff1f1040f29 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:16.819129 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d3e9cc-03da-4d41-b13a-8ff1f1040f29 None admin] PATH_INFO: `/v3/projects/54952dee2ab9420d9541e99a1825f5af` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:16.820281 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d3e9cc-03da-4d41-b13a-8ff1f1040f29 None admin] RBAC: Authorizing `identity:get_project(project_id=54952dee2ab9420d9541e99a1825f5af)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:16.821734 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d3e9cc-03da-4d41-b13a-8ff1f1040f29 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:16.823398 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 96/196] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Aug 25 03:00:16 2022] GET /identity/v3/projects/54952dee2ab9420d9541e99a1825f5af => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:16.824157 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d3e9cc-03da-4d41-b13a-8ff1f1040f29 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:16.891110 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:16.891604 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a80e5a4-9264-4b2d-a738-f8b61d99a5c0 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:16.901688 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:16.923164 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d3e9cc-03da-4d41-b13a-8ff1f1040f29 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:16.924877 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c06e0c1-e5a4-4697-8570-ab87611d83ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:16.925481 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c06e0c1-e5a4-4697-8570-ab87611d83ff None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:16.925757 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c06e0c1-e5a4-4697-8570-ab87611d83ff None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:16.926028 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c06e0c1-e5a4-4697-8570-ab87611d83ff None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:16.926724 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c06e0c1-e5a4-4697-8570-ab87611d83ff None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:16.927025 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c06e0c1-e5a4-4697-8570-ab87611d83ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:16.928390 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c06e0c1-e5a4-4697-8570-ab87611d83ff None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:16.937589 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 101/197] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Aug 25 03:00:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:16.938793 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c06e0c1-e5a4-4697-8570-ab87611d83ff None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:16.962619 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342eff29-1a9e-471d-a2b1-643d21e7072d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:16.963983 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342eff29-1a9e-471d-a2b1-643d21e7072d None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:16.963983 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342eff29-1a9e-471d-a2b1-643d21e7072d None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:16.963983 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342eff29-1a9e-471d-a2b1-643d21e7072d None admin] PATH_INFO: `/v3/users/064052b980324793a0bc0266db01c33b` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:16.965065 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342eff29-1a9e-471d-a2b1-643d21e7072d None admin] RBAC: Authorizing `identity:get_user(user_id=064052b980324793a0bc0266db01c33b)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:16.966544 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342eff29-1a9e-471d-a2b1-643d21e7072d None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:16.968279 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 97/198] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Aug 25 03:00:16 2022] GET /identity/v3/users/064052b980324793a0bc0266db01c33b => generated 289 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:16.969053 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342eff29-1a9e-471d-a2b1-643d21e7072d None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:17.038814 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c06e0c1-e5a4-4697-8570-ab87611d83ff None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:17.067603 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342eff29-1a9e-471d-a2b1-643d21e7072d None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:17.072044 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b5a646-0d85-47b0-b81e-523a3fa6869b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:17.072651 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b5a646-0d85-47b0-b81e-523a3fa6869b None admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:17.072926 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b5a646-0d85-47b0-b81e-523a3fa6869b None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:17.073194 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b5a646-0d85-47b0-b81e-523a3fa6869b None admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:17.073830 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b5a646-0d85-47b0-b81e-523a3fa6869b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:17.075279 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b5a646-0d85-47b0-b81e-523a3fa6869b None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:17.100715 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c410ca0e-5e2a-4457-ac1a-0ec32a94e4be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:17.102769 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c410ca0e-5e2a-4457-ac1a-0ec32a94e4be None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:17.102769 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c410ca0e-5e2a-4457-ac1a-0ec32a94e4be None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:17.102769 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c410ca0e-5e2a-4457-ac1a-0ec32a94e4be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:17.102769 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c410ca0e-5e2a-4457-ac1a-0ec32a94e4be None admin] RBAC: Adding query filter params (role.id=3255f77192524a16bc405173106871fc, scope.project.id=4a3808105deb490daa12dd1f297e4b06, user.id=26e7c9fc5d3c41de8156071d9f0cb3cd) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:17.103085 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c410ca0e-5e2a-4457-ac1a-0ec32a94e4be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:17.104302 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c410ca0e-5e2a-4457-ac1a-0ec32a94e4be None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:17.107056 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:17.110671 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 98/199] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Aug 25 03:00:16 2022] GET /identity/v3/role_assignments?user.id=26e7c9fc5d3c41de8156071d9f0cb3cd&scope.project.id=4a3808105deb490daa12dd1f297e4b06&role.id=3255f77192524a16bc405173106871fc => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:17.111528 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c410ca0e-5e2a-4457-ac1a-0ec32a94e4be None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:17.160409 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 102/200] 10.222.0.58 () {60 vars in 1132 bytes} [Thu Aug 25 03:00:16 2022] POST /identity/v3/users => generated 297 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:00:17.161368 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b5a646-0d85-47b0-b81e-523a3fa6869b None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:17.202526 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:17.233651 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c410ca0e-5e2a-4457-ac1a-0ec32a94e4be None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:17.280161 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84bc5d2-2860-426d-9629-2b7683788171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:17.281041 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84bc5d2-2860-426d-9629-2b7683788171 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:17.282093 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84bc5d2-2860-426d-9629-2b7683788171 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:17.282400 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84bc5d2-2860-426d-9629-2b7683788171 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:17.283339 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84bc5d2-2860-426d-9629-2b7683788171 None admin] RBAC: Adding query filter params (role.id=62f19ea272b447ccaab1576cd9bf4fd9, scope.project.id=54952dee2ab9420d9541e99a1825f5af, user.id=90cfb8b65288490aadfab2795371d6e2) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:17.283557 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84bc5d2-2860-426d-9629-2b7683788171 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:17.285100 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84bc5d2-2860-426d-9629-2b7683788171 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:17.292218 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 99/201] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Aug 25 03:00:17 2022] GET /identity/v3/role_assignments?user.id=90cfb8b65288490aadfab2795371d6e2&scope.project.id=54952dee2ab9420d9541e99a1825f5af&role.id=62f19ea272b447ccaab1576cd9bf4fd9 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:17.292890 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84bc5d2-2860-426d-9629-2b7683788171 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:17.319305 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b5a646-0d85-47b0-b81e-523a3fa6869b None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:17.362532 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9db465-99cb-41b1-a04c-64ea0b0d62c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:17.363286 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9db465-99cb-41b1-a04c-64ea0b0d62c9 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:17.363634 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9db465-99cb-41b1-a04c-64ea0b0d62c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:17.363977 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9db465-99cb-41b1-a04c-64ea0b0d62c9 None admin] PATH_INFO: `/v3/users/90cfb8b65288490aadfab2795371d6e2` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:17.365258 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9db465-99cb-41b1-a04c-64ea0b0d62c9 None admin] RBAC: Authorizing `identity:get_user(user_id=90cfb8b65288490aadfab2795371d6e2)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:17.367850 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9db465-99cb-41b1-a04c-64ea0b0d62c9 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:17.369790 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 103/202] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Aug 25 03:00:17 2022] GET /identity/v3/users/90cfb8b65288490aadfab2795371d6e2 => generated 283 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:17.375603 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:17.376271 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9db465-99cb-41b1-a04c-64ea0b0d62c9 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:17.446764 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84bc5d2-2860-426d-9629-2b7683788171 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:17.487816 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0939d0-c31c-434b-96da-68e9d9ef52e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:17.488438 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0939d0-c31c-434b-96da-68e9d9ef52e4 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:17.489029 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0939d0-c31c-434b-96da-68e9d9ef52e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:17.489305 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0939d0-c31c-434b-96da-68e9d9ef52e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:17.514454 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:17.521737 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9db465-99cb-41b1-a04c-64ea0b0d62c9 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:17.574963 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0939d0-c31c-434b-96da-68e9d9ef52e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:17.576446 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0939d0-c31c-434b-96da-68e9d9ef52e4 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:17.580466 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c53c8f-486c-4521-8804-ba474af98198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:17.580904 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c53c8f-486c-4521-8804-ba474af98198 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:17.581185 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c53c8f-486c-4521-8804-ba474af98198 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:17.581454 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c53c8f-486c-4521-8804-ba474af98198 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:17.676567 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c53c8f-486c-4521-8804-ba474af98198 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:17.678026 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c53c8f-486c-4521-8804-ba474af98198 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:17.683959 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:17.735478 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 100/203] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Aug 25 03:00:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 25 03:00:17.741739 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0939d0-c31c-434b-96da-68e9d9ef52e4 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:17.838675 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:17.841191 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 104/204] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Aug 25 03:00:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 25 03:00:17.842572 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c53c8f-486c-4521-8804-ba474af98198 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:17.956411 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c53c8f-486c-4521-8804-ba474af98198 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:17.968062 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0939d0-c31c-434b-96da-68e9d9ef52e4 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:17.991890 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b183aea1-f4fd-48b2-8a41-e76dd30bcfee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:17.992482 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b183aea1-f4fd-48b2-8a41-e76dd30bcfee None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:17.992766 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b183aea1-f4fd-48b2-8a41-e76dd30bcfee None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:17.993044 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b183aea1-f4fd-48b2-8a41-e76dd30bcfee None admin] PATH_INFO: `/v3/projects/54952dee2ab9420d9541e99a1825f5af` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:17.994668 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b183aea1-f4fd-48b2-8a41-e76dd30bcfee None admin] RBAC: Authorizing `identity:get_project(project_id=54952dee2ab9420d9541e99a1825f5af)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:17.996226 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b183aea1-f4fd-48b2-8a41-e76dd30bcfee None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:18.002394 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 105/205] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Aug 25 03:00:17 2022] GET /identity/v3/projects/54952dee2ab9420d9541e99a1825f5af => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:18.004691 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b183aea1-f4fd-48b2-8a41-e76dd30bcfee None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:18.011988 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e58f35-879e-423f-ae38-ef0bf12d308f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:18.012827 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e58f35-879e-423f-ae38-ef0bf12d308f None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:18.013253 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e58f35-879e-423f-ae38-ef0bf12d308f None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:18.013696 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e58f35-879e-423f-ae38-ef0bf12d308f None admin] PATH_INFO: `/v3/projects/0d32389226ec4d2dac8c390a33d5436c` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:18.015269 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e58f35-879e-423f-ae38-ef0bf12d308f None admin] RBAC: Authorizing `identity:get_project(project_id=0d32389226ec4d2dac8c390a33d5436c)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:18.016992 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e58f35-879e-423f-ae38-ef0bf12d308f None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:18.018955 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 101/206] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Aug 25 03:00:17 2022] GET /identity/v3/projects/0d32389226ec4d2dac8c390a33d5436c => generated 298 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:00:18.031376 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e58f35-879e-423f-ae38-ef0bf12d308f None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:18.069229 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:18.103667 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:18.106835 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b183aea1-f4fd-48b2-8a41-e76dd30bcfee None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:18.144748 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdafa526-1502-423b-833d-10d309f8f6a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:18.145370 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdafa526-1502-423b-833d-10d309f8f6a4 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:18.145658 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdafa526-1502-423b-833d-10d309f8f6a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:18.145943 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdafa526-1502-423b-833d-10d309f8f6a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:18.146717 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdafa526-1502-423b-833d-10d309f8f6a4 None admin] RBAC: Adding query filter params (role.id=7886cc7e333a42d08085932494a3ea6b, scope.project.id=54952dee2ab9420d9541e99a1825f5af, user.id=90cfb8b65288490aadfab2795371d6e2) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:18.147032 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdafa526-1502-423b-833d-10d309f8f6a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:18.149087 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdafa526-1502-423b-833d-10d309f8f6a4 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:18.155543 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 106/207] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Aug 25 03:00:18 2022] GET /identity/v3/role_assignments?user.id=90cfb8b65288490aadfab2795371d6e2&scope.project.id=54952dee2ab9420d9541e99a1825f5af&role.id=7886cc7e333a42d08085932494a3ea6b => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:00:18.241542 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e58f35-879e-423f-ae38-ef0bf12d308f None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:18.303650 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2974401-6f65-4091-b7fd-46e911e97d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:18.304669 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2974401-6f65-4091-b7fd-46e911e97d51 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:18.305150 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2974401-6f65-4091-b7fd-46e911e97d51 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:18.305622 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2974401-6f65-4091-b7fd-46e911e97d51 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:18.306600 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2974401-6f65-4091-b7fd-46e911e97d51 None admin] RBAC: Adding query filter params (role.id=3de9d2fd0b464c399f67fb5240905596, scope.project.id=0d32389226ec4d2dac8c390a33d5436c, user.id=064052b980324793a0bc0266db01c33b) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:18.307130 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2974401-6f65-4091-b7fd-46e911e97d51 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:18.308779 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2974401-6f65-4091-b7fd-46e911e97d51 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:18.319176 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 102/208] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Aug 25 03:00:18 2022] GET /identity/v3/role_assignments?user.id=064052b980324793a0bc0266db01c33b&scope.project.id=0d32389226ec4d2dac8c390a33d5436c&role.id=3de9d2fd0b464c399f67fb5240905596 => generated 608 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:19.410886 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4561196d-f0bc-46ab-bc0f-6ef79521dc7f None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:19.411433 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4561196d-f0bc-46ab-bc0f-6ef79521dc7f None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:19.411433 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4561196d-f0bc-46ab-bc0f-6ef79521dc7f None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:19.413033 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 107/209] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:00:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:00:19.418080 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9068da37-fc36-4cce-b379-704f2e160001 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:19.418494 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9068da37-fc36-4cce-b379-704f2e160001 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:19.418661 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9068da37-fc36-4cce-b379-704f2e160001 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:19.437949 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-9068da37-fc36-4cce-b379-704f2e160001 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:00:19.439147 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-9068da37-fc36-4cce-b379-704f2e160001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:19.485116 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352813aa-cd69-4349-a20f-34d057194306 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:19.485723 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352813aa-cd69-4349-a20f-34d057194306 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:19.486218 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352813aa-cd69-4349-a20f-34d057194306 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:19.488047 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 108/210] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:00:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 25 03:00:19.493744 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb8197b-4e9a-4ca8-9a5d-b8cf7b9e02cc None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:19.494554 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb8197b-4e9a-4ca8-9a5d-b8cf7b9e02cc None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:19.495033 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb8197b-4e9a-4ca8-9a5d-b8cf7b9e02cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:19.515989 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 103/211] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:00:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 03:00:19.517808 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-3bb8197b-4e9a-4ca8-9a5d-b8cf7b9e02cc None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:00:19.522347 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-3bb8197b-4e9a-4ca8-9a5d-b8cf7b9e02cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:19.590093 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 109/212] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:00:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 03:00:19.791529 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9068da37-fc36-4cce-b379-704f2e160001 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:19.818648 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:19.821382 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec683664-14a8-4fa2-8648-785b15975bcc None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:19.821663 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec683664-14a8-4fa2-8648-785b15975bcc None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:19.821936 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec683664-14a8-4fa2-8648-785b15975bcc None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:19.823591 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 110/213] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:00:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:00:19.833554 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb134766-198b-44ba-bde3-e4233a1b3056 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:19.840821 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb134766-198b-44ba-bde3-e4233a1b3056 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:19.840821 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb134766-198b-44ba-bde3-e4233a1b3056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:19.857120 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-eb134766-198b-44ba-bde3-e4233a1b3056 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:00:19.859873 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-eb134766-198b-44ba-bde3-e4233a1b3056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:19.903744 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9068da37-fc36-4cce-b379-704f2e160001 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:19.922273 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:19.936838 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 111/214] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:00:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 25 03:00:19.940824 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf3e5af-ac2c-421b-bf74-8e618ed95404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:19.941738 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf3e5af-ac2c-421b-bf74-8e618ed95404 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:19.942573 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf3e5af-ac2c-421b-bf74-8e618ed95404 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:19.943068 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf3e5af-ac2c-421b-bf74-8e618ed95404 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:19.943170 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb134766-198b-44ba-bde3-e4233a1b3056 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:19.954790 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf3e5af-ac2c-421b-bf74-8e618ed95404 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:00:19.959910 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-ccf3e5af-ac2c-421b-bf74-8e618ed95404 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 03:00:19.959910 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:00:19.959910 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:00:19.959910 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:00:19.959910 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:00:19.959910 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:00:19.959910 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:00:19.959910 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:00:19.959910 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:00:19.959910 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:00:19.959910 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:00:19.959910 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:00:19.959910 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:00:19.959910 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:00:19.959910 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:00:19.959910 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:00:19.960794 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:00:19.960794 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:00:19.960794 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:00:19.960794 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:00:19.960794 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:00:19.960794 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:00:19.960794 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:00:19.960794 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:00:19.960794 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:00:19.960794 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:00:19.960794 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:00:19.960794 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:00:19.960794 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:00:19.960794 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:00:19.960794 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:00:19.960794 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:00:19.961586 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:00:19.961586 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:00:19.961586 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:00:19.961586 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:00:19.961586 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:00:19.961586 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:00:19.961586 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:00:19.961586 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:00:19.961586 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:00:19.961586 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:00:19.961586 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 03:00:19.961586 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:00:19.963803 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 104/215] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Aug 25 03:00:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:00:19.970434 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf3e5af-ac2c-421b-bf74-8e618ed95404 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:20.069694 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:20.071761 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb134766-198b-44ba-bde3-e4233a1b3056 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:20.086063 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf3e5af-ac2c-421b-bf74-8e618ed95404 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:20.117437 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f048a70-8056-4be7-aa0b-e7e0ead27854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:20.118362 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f048a70-8056-4be7-aa0b-e7e0ead27854 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:20.118645 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f048a70-8056-4be7-aa0b-e7e0ead27854 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:20.118918 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f048a70-8056-4be7-aa0b-e7e0ead27854 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:20.125811 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cab6bd-97f5-46e0-99e3-93708d031780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:20.126695 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cab6bd-97f5-46e0-99e3-93708d031780 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:20.127240 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cab6bd-97f5-46e0-99e3-93708d031780 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:20.127819 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cab6bd-97f5-46e0-99e3-93708d031780 None admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:20.128680 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cab6bd-97f5-46e0-99e3-93708d031780 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:20.129175 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cab6bd-97f5-46e0-99e3-93708d031780 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:20.130469 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f048a70-8056-4be7-aa0b-e7e0ead27854 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:00:20.131160 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cab6bd-97f5-46e0-99e3-93708d031780 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:20.135863 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-8f048a70-8056-4be7-aa0b-e7e0ead27854 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 25 03:00:20.135863 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:00:20.135863 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:00:20.135863 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:00:20.135863 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:00:20.135863 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:00:20.135863 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:00:20.135863 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:00:20.135863 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:00:20.135863 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:00:20.135863 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:00:20.135863 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:00:20.135863 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:00:20.135863 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:00:20.135863 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:00:20.135863 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:00:20.136719 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:00:20.136719 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:00:20.136719 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:00:20.136719 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:00:20.136719 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:00:20.136719 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:00:20.136719 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:00:20.136719 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:00:20.136719 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:00:20.136719 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:00:20.136719 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:00:20.136719 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:00:20.136719 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:00:20.136719 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:00:20.136719 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:00:20.136719 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:00:20.137497 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:00:20.137497 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:00:20.137497 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:00:20.137497 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:00:20.137497 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:00:20.137497 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:00:20.137497 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:00:20.137497 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:00:20.137497 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:00:20.137497 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:00:20.137497 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 25 03:00:20.137497 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:00:20.137497 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 112/216] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Aug 25 03:00:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:00:20.146388 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 105/217] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Aug 25 03:00:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:20.148784 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f048a70-8056-4be7-aa0b-e7e0ead27854 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:20.148897 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cab6bd-97f5-46e0-99e3-93708d031780 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:20.208783 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:20.255122 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cab6bd-97f5-46e0-99e3-93708d031780 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:20.257782 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f048a70-8056-4be7-aa0b-e7e0ead27854 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:20.288247 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21423f3-c58f-4911-9702-7bb0033cc054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:20.288880 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21423f3-c58f-4911-9702-7bb0033cc054 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:20.289158 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21423f3-c58f-4911-9702-7bb0033cc054 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:20.289431 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21423f3-c58f-4911-9702-7bb0033cc054 None admin] PATH_INFO: `/v3/users/90cfb8b65288490aadfab2795371d6e2` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:20.290576 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21423f3-c58f-4911-9702-7bb0033cc054 None admin] RBAC: Authorizing `identity:get_user(user_id=90cfb8b65288490aadfab2795371d6e2)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:20.291640 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a531c23-10e9-4f61-b66b-1d46a0b7c457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:20.291985 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21423f3-c58f-4911-9702-7bb0033cc054 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:20.292246 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a531c23-10e9-4f61-b66b-1d46a0b7c457 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:20.292529 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a531c23-10e9-4f61-b66b-1d46a0b7c457 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:20.292806 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a531c23-10e9-4f61-b66b-1d46a0b7c457 None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:20.293474 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a531c23-10e9-4f61-b66b-1d46a0b7c457 None admin] RBAC: Adding query filter params (name=member) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:20.293572 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 106/218] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Aug 25 03:00:20 2022] GET /identity/v3/users/90cfb8b65288490aadfab2795371d6e2 => generated 283 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:20.293856 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a531c23-10e9-4f61-b66b-1d46a0b7c457 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:20.295324 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a531c23-10e9-4f61-b66b-1d46a0b7c457 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:20.295653 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3733aca9-db7c-4b2d-b656-4785c1303069 None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:20.295945 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3733aca9-db7c-4b2d-b656-4785c1303069 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:20.296227 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3733aca9-db7c-4b2d-b656-4785c1303069 None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:20.297853 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 107/219] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:00:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:00:20.299023 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3733aca9-db7c-4b2d-b656-4785c1303069 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:20.303763 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 113/220] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Aug 25 03:00:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:20.304619 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a531c23-10e9-4f61-b66b-1d46a0b7c457 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:20.319502 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:20.401779 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3733aca9-db7c-4b2d-b656-4785c1303069 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:20.404068 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a531c23-10e9-4f61-b66b-1d46a0b7c457 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:20.415130 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:20.435036 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f671b6bc-d56d-4cfa-bb01-9733f7261424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:20.435639 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f671b6bc-d56d-4cfa-bb01-9733f7261424 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:20.435932 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f671b6bc-d56d-4cfa-bb01-9733f7261424 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:20.436202 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f671b6bc-d56d-4cfa-bb01-9733f7261424 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:20.437284 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f671b6bc-d56d-4cfa-bb01-9733f7261424 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:20.437426 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be56813-c204-4730-953d-5b006e177aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:20.438101 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be56813-c204-4730-953d-5b006e177aea None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:20.438453 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be56813-c204-4730-953d-5b006e177aea None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:20.438842 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f671b6bc-d56d-4cfa-bb01-9733f7261424 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:20.438986 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be56813-c204-4730-953d-5b006e177aea None admin] PATH_INFO: `/v3/projects/54952dee2ab9420d9541e99a1825f5af` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:20.440565 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 108/221] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:00:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:20.440682 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be56813-c204-4730-953d-5b006e177aea None admin] RBAC: Authorizing `identity:get_project(project_id=54952dee2ab9420d9541e99a1825f5af)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:20.441501 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bd9375-5e9e-47a0-8193-bccc9ef501ed None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:20.441786 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bd9375-5e9e-47a0-8193-bccc9ef501ed None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:20.441918 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be56813-c204-4730-953d-5b006e177aea None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:20.442064 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bd9375-5e9e-47a0-8193-bccc9ef501ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:20.448267 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 114/222] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Aug 25 03:00:20 2022] GET /identity/v3/projects/54952dee2ab9420d9541e99a1825f5af => generated 302 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:20.448267 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be56813-c204-4730-953d-5b006e177aea None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:20.463380 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-99bd9375-5e9e-47a0-8193-bccc9ef501ed None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:00:20.464570 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-99bd9375-5e9e-47a0-8193-bccc9ef501ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:20.529981 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 109/223] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:00:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 03:00:20.530810 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bd9375-5e9e-47a0-8193-bccc9ef501ed None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:20.542630 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be56813-c204-4730-953d-5b006e177aea None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:20.558728 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:20.576774 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37db0db-40b6-45e4-b742-79b0fc360a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:20.578746 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37db0db-40b6-45e4-b742-79b0fc360a5a None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:20.579025 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37db0db-40b6-45e4-b742-79b0fc360a5a None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:20.579337 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37db0db-40b6-45e4-b742-79b0fc360a5a None admin] PATH_INFO: `/v3/users/87b51680f137409c97169ee267d279e5` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:20.593712 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37db0db-40b6-45e4-b742-79b0fc360a5a None admin] RBAC: Authorizing `identity:get_user(user_id=87b51680f137409c97169ee267d279e5)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:20.595220 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37db0db-40b6-45e4-b742-79b0fc360a5a None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:20.596872 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 115/224] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Aug 25 03:00:20 2022] GET /identity/v3/users/87b51680f137409c97169ee267d279e5 => generated 297 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:20.602128 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37db0db-40b6-45e4-b742-79b0fc360a5a None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:20.642602 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:20.663530 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bd9375-5e9e-47a0-8193-bccc9ef501ed None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:20.700572 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a2a085-496d-489a-a055-fe6ce49c748a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:20.701395 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a2a085-496d-489a-a055-fe6ce49c748a None admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:20.701753 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a2a085-496d-489a-a055-fe6ce49c748a None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:20.702934 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a2a085-496d-489a-a055-fe6ce49c748a None admin] PATH_INFO: `/v3/users` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:20.703570 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a2a085-496d-489a-a055-fe6ce49c748a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:20.705005 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a2a085-496d-489a-a055-fe6ce49c748a None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:20.709527 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37db0db-40b6-45e4-b742-79b0fc360a5a None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:20.743695 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7679dcd4-39bf-4529-a54d-e525348ff201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:20.744339 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7679dcd4-39bf-4529-a54d-e525348ff201 None admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:20.744618 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7679dcd4-39bf-4529-a54d-e525348ff201 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:20.744893 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7679dcd4-39bf-4529-a54d-e525348ff201 None admin] PATH_INFO: `/v3/projects/54952dee2ab9420d9541e99a1825f5af/users/90cfb8b65288490aadfab2795371d6e2/roles/62f19ea272b447ccaab1576cd9bf4fd9` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:20.746788 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7679dcd4-39bf-4529-a54d-e525348ff201 None admin] RBAC: Authorizing `identity:create_grant(project_id=54952dee2ab9420d9541e99a1825f5af, user_id=90cfb8b65288490aadfab2795371d6e2, role_id=62f19ea272b447ccaab1576cd9bf4fd9)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:20.748209 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7679dcd4-39bf-4529-a54d-e525348ff201 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:20.793303 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 110/225] 10.222.0.58 () {60 vars in 1132 bytes} [Thu Aug 25 03:00:20 2022] POST /identity/v3/users => generated 297 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:00:20.794132 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a2a085-496d-489a-a055-fe6ce49c748a None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:20.827788 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 116/226] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Aug 25 03:00:20 2022] PUT /identity/v3/projects/54952dee2ab9420d9541e99a1825f5af/users/90cfb8b65288490aadfab2795371d6e2/roles/62f19ea272b447ccaab1576cd9bf4fd9 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:00:20.828537 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7679dcd4-39bf-4529-a54d-e525348ff201 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:20.898230 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:20.937879 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a2a085-496d-489a-a055-fe6ce49c748a None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:20.983552 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:20.992013 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7679dcd4-39bf-4529-a54d-e525348ff201 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:20.993323 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894d5173-c1ff-45c9-bce7-e927fea1905b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:20.994420 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894d5173-c1ff-45c9-bce7-e927fea1905b None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:20.994758 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894d5173-c1ff-45c9-bce7-e927fea1905b None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:20.995042 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894d5173-c1ff-45c9-bce7-e927fea1905b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:21.034782 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218b14e6-c48f-49b9-b981-4acad942ae89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:21.035432 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218b14e6-c48f-49b9-b981-4acad942ae89 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:21.035712 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218b14e6-c48f-49b9-b981-4acad942ae89 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:21.035988 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218b14e6-c48f-49b9-b981-4acad942ae89 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:21.047384 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218b14e6-c48f-49b9-b981-4acad942ae89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:00:21.052125 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-218b14e6-c48f-49b9-b981-4acad942ae89 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 25 03:00:21.052125 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:00:21.052125 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:00:21.052125 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:00:21.052125 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:00:21.052125 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:00:21.052125 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:00:21.052125 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:00:21.052125 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:00:21.052125 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:00:21.052125 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:00:21.052125 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:00:21.052125 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:00:21.052125 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:00:21.052125 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:00:21.052125 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:00:21.053001 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:00:21.053001 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:00:21.053001 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:00:21.053001 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:00:21.053001 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:00:21.053001 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:00:21.053001 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:00:21.053001 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:00:21.053001 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:00:21.053001 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:00:21.053001 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:00:21.053001 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:00:21.053001 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:00:21.053001 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:00:21.053001 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:00:21.053001 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:00:21.053783 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:00:21.053783 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:00:21.053783 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:00:21.053783 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:00:21.053783 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:00:21.053783 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:00:21.053783 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:00:21.053783 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:00:21.053783 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:00:21.053783 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:00:21.053783 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 25 03:00:21.053783 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:00:21.053783 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 117/227] 10.222.0.58 () {56 vars in 1114 bytes} [Thu Aug 25 03:00:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:00:21.059356 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218b14e6-c48f-49b9-b981-4acad942ae89 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:21.132219 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894d5173-c1ff-45c9-bce7-e927fea1905b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:21.133784 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894d5173-c1ff-45c9-bce7-e927fea1905b None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:21.204460 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218b14e6-c48f-49b9-b981-4acad942ae89 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:21.231299 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:21.245430 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d338c3-a0a0-4ec6-bcb3-8ab90180c74d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:21.246047 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d338c3-a0a0-4ec6-bcb3-8ab90180c74d None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:21.247151 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d338c3-a0a0-4ec6-bcb3-8ab90180c74d None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:21.247151 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d338c3-a0a0-4ec6-bcb3-8ab90180c74d None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:21.247316 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d338c3-a0a0-4ec6-bcb3-8ab90180c74d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:21.247568 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d338c3-a0a0-4ec6-bcb3-8ab90180c74d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:21.249002 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d338c3-a0a0-4ec6-bcb3-8ab90180c74d None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:21.264325 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 118/228] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Aug 25 03:00:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:21.265907 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d338c3-a0a0-4ec6-bcb3-8ab90180c74d None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:21.280476 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 111/229] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Aug 25 03:00:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 25 03:00:21.287080 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894d5173-c1ff-45c9-bce7-e927fea1905b None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:21.302692 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:21.421250 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d338c3-a0a0-4ec6-bcb3-8ab90180c74d None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:21.429302 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894d5173-c1ff-45c9-bce7-e927fea1905b None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:21.485705 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afcdb93-ada5-4011-ad3e-5246de804d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:21.486291 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afcdb93-ada5-4011-ad3e-5246de804d14 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:21.488024 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afcdb93-ada5-4011-ad3e-5246de804d14 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:21.488308 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afcdb93-ada5-4011-ad3e-5246de804d14 None admin] PATH_INFO: `/v3/users/90cfb8b65288490aadfab2795371d6e2` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:21.489772 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afcdb93-ada5-4011-ad3e-5246de804d14 None admin] RBAC: Authorizing `identity:get_user(user_id=90cfb8b65288490aadfab2795371d6e2)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:21.491780 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afcdb93-ada5-4011-ad3e-5246de804d14 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:21.493449 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 119/230] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Aug 25 03:00:21 2022] GET /identity/v3/users/90cfb8b65288490aadfab2795371d6e2 => generated 283 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:21.500664 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4928f8-df78-4cf2-ab57-dcee977cef86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:21.501521 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4928f8-df78-4cf2-ab57-dcee977cef86 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:21.501810 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4928f8-df78-4cf2-ab57-dcee977cef86 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:21.502092 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4928f8-df78-4cf2-ab57-dcee977cef86 None admin] PATH_INFO: `/v3/projects/54952dee2ab9420d9541e99a1825f5af` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:21.503247 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afcdb93-ada5-4011-ad3e-5246de804d14 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:21.503415 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4928f8-df78-4cf2-ab57-dcee977cef86 None admin] RBAC: Authorizing `identity:get_project(project_id=54952dee2ab9420d9541e99a1825f5af)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:21.504760 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4928f8-df78-4cf2-ab57-dcee977cef86 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:21.506768 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 112/231] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Aug 25 03:00:21 2022] GET /identity/v3/projects/54952dee2ab9420d9541e99a1825f5af => generated 302 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:21.519783 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4928f8-df78-4cf2-ab57-dcee977cef86 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:21.610873 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:21.614292 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:21.634120 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afcdb93-ada5-4011-ad3e-5246de804d14 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:21.687035 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223dbcfa-f942-4f9d-a19f-96368c630be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:21.687698 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223dbcfa-f942-4f9d-a19f-96368c630be3 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:21.687984 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223dbcfa-f942-4f9d-a19f-96368c630be3 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:21.688262 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223dbcfa-f942-4f9d-a19f-96368c630be3 None admin] PATH_INFO: `/v3/projects/54952dee2ab9420d9541e99a1825f5af` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:21.690952 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223dbcfa-f942-4f9d-a19f-96368c630be3 None admin] RBAC: Authorizing `identity:get_project(project_id=54952dee2ab9420d9541e99a1825f5af)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:21.692417 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223dbcfa-f942-4f9d-a19f-96368c630be3 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:21.694395 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 120/232] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Aug 25 03:00:21 2022] GET /identity/v3/projects/54952dee2ab9420d9541e99a1825f5af => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:21.699848 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223dbcfa-f942-4f9d-a19f-96368c630be3 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:21.734843 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4928f8-df78-4cf2-ab57-dcee977cef86 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:21.808248 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911d9885-bd57-4516-922e-517b37def215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:21.810492 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911d9885-bd57-4516-922e-517b37def215 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:21.811968 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911d9885-bd57-4516-922e-517b37def215 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:21.812470 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911d9885-bd57-4516-922e-517b37def215 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:21.813416 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911d9885-bd57-4516-922e-517b37def215 None admin] RBAC: Adding query filter params (role.id=3255f77192524a16bc405173106871fc, scope.project.id=54952dee2ab9420d9541e99a1825f5af, user.id=87b51680f137409c97169ee267d279e5) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:21.815176 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911d9885-bd57-4516-922e-517b37def215 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:21.816846 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911d9885-bd57-4516-922e-517b37def215 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:21.830406 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 113/233] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Aug 25 03:00:21 2022] GET /identity/v3/role_assignments?user.id=87b51680f137409c97169ee267d279e5&scope.project.id=54952dee2ab9420d9541e99a1825f5af&role.id=3255f77192524a16bc405173106871fc => generated 259 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:21.873104 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223dbcfa-f942-4f9d-a19f-96368c630be3 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:21.911140 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf263fb6-b456-4700-b661-04308de73140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:21.912171 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf263fb6-b456-4700-b661-04308de73140 None admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:21.912655 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf263fb6-b456-4700-b661-04308de73140 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:21.913155 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf263fb6-b456-4700-b661-04308de73140 None admin] PATH_INFO: `/v3/projects/54952dee2ab9420d9541e99a1825f5af/users/90cfb8b65288490aadfab2795371d6e2/roles/7886cc7e333a42d08085932494a3ea6b` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:21.915731 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf263fb6-b456-4700-b661-04308de73140 None admin] RBAC: Authorizing `identity:create_grant(project_id=54952dee2ab9420d9541e99a1825f5af, user_id=90cfb8b65288490aadfab2795371d6e2, role_id=7886cc7e333a42d08085932494a3ea6b)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:21.917495 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf263fb6-b456-4700-b661-04308de73140 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:22.033696 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 121/234] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Aug 25 03:00:21 2022] PUT /identity/v3/projects/54952dee2ab9420d9541e99a1825f5af/users/90cfb8b65288490aadfab2795371d6e2/roles/7886cc7e333a42d08085932494a3ea6b => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:00:23.206273 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cd0cf4-50cf-429b-abd2-d804be2e03be None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:23.206837 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cd0cf4-50cf-429b-abd2-d804be2e03be None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:23.206837 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cd0cf4-50cf-429b-abd2-d804be2e03be None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:23.208381 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 114/235] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:00:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:00:23.213910 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98b1511-d0c4-48d3-8eda-825c6a7f76ce None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:23.214323 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98b1511-d0c4-48d3-8eda-825c6a7f76ce None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:23.214519 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98b1511-d0c4-48d3-8eda-825c6a7f76ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:23.239287 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-a98b1511-d0c4-48d3-8eda-825c6a7f76ce None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:00:23.240513 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-a98b1511-d0c4-48d3-8eda-825c6a7f76ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:23.251106 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:23.329394 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c178e0f8-6e1d-4fbf-8d5a-8f98429f7da8 None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:23.329716 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c178e0f8-6e1d-4fbf-8d5a-8f98429f7da8 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:23.329949 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c178e0f8-6e1d-4fbf-8d5a-8f98429f7da8 None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:23.331248 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 122/236] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:00:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 03:00:23.334584 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 115/237] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:00:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:00:23.353410 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341c7e20-569d-4647-a80e-102bb58536cb None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:23.353709 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341c7e20-569d-4647-a80e-102bb58536cb None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:23.353974 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341c7e20-569d-4647-a80e-102bb58536cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:23.375270 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-341c7e20-569d-4647-a80e-102bb58536cb None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:00:23.376784 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-341c7e20-569d-4647-a80e-102bb58536cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:23.388266 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714389a6-87ce-49cf-bf8a-0c09f3a2e09e None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:23.388711 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714389a6-87ce-49cf-bf8a-0c09f3a2e09e None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:23.388839 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714389a6-87ce-49cf-bf8a-0c09f3a2e09e None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:23.390827 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 116/238] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:00:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:00:23.395793 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a4a42c-d2d3-4341-961b-96a5d8ab36fe None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:23.396076 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a4a42c-d2d3-4341-961b-96a5d8ab36fe None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:23.396351 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a4a42c-d2d3-4341-961b-96a5d8ab36fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:23.414710 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-a5a4a42c-d2d3-4341-961b-96a5d8ab36fe None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:00:23.415893 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-a5a4a42c-d2d3-4341-961b-96a5d8ab36fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:23.508468 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 123/239] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:00:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 03:00:23.516190 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 117/240] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:00:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 25 03:00:23.723114 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341c7e20-569d-4647-a80e-102bb58536cb None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:23.812037 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a4a42c-d2d3-4341-961b-96a5d8ab36fe None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:23.825215 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:23.843428 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:23.854269 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341c7e20-569d-4647-a80e-102bb58536cb None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:23.899595 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5db185-a9ca-4ddf-9f34-51142adc4571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:23.900295 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5db185-a9ca-4ddf-9f34-51142adc4571 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:23.900584 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5db185-a9ca-4ddf-9f34-51142adc4571 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:23.900865 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5db185-a9ca-4ddf-9f34-51142adc4571 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:23.914391 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a4a42c-d2d3-4341-961b-96a5d8ab36fe None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:23.923648 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5db185-a9ca-4ddf-9f34-51142adc4571 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:00:23.928474 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-8f5db185-a9ca-4ddf-9f34-51142adc4571 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 03:00:23.928474 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:00:23.928474 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:00:23.928474 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:00:23.928474 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:00:23.928474 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:00:23.928474 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:00:23.928474 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:00:23.928474 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:00:23.928474 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:00:23.928474 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:00:23.928474 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:00:23.928474 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:00:23.928474 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:00:23.928474 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:00:23.928474 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:00:23.929351 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:00:23.929351 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:00:23.929351 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:00:23.929351 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:00:23.929351 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:00:23.929351 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:00:23.929351 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:00:23.929351 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:00:23.929351 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:00:23.929351 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:00:23.929351 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:00:23.929351 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:00:23.929351 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:00:23.929351 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:00:23.929351 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:00:23.929351 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:00:23.930139 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:00:23.930139 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:00:23.930139 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:00:23.930139 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:00:23.930139 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:00:23.930139 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:00:23.930139 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:00:23.930139 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:00:23.930139 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:00:23.930139 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:00:23.930139 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 03:00:23.930139 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:00:23.930139 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 124/241] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Aug 25 03:00:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:00:23.934877 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5db185-a9ca-4ddf-9f34-51142adc4571 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:23.950784 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dcb201-4622-4e38-8dcb-4b4ba21dac03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:23.951561 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dcb201-4622-4e38-8dcb-4b4ba21dac03 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:23.952430 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dcb201-4622-4e38-8dcb-4b4ba21dac03 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:23.952705 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dcb201-4622-4e38-8dcb-4b4ba21dac03 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:23.964105 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dcb201-4622-4e38-8dcb-4b4ba21dac03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:00:23.969898 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-73dcb201-4622-4e38-8dcb-4b4ba21dac03 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 25 03:00:23.969898 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:00:23.969898 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:00:23.969898 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:00:23.969898 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:00:23.969898 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:00:23.969898 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:00:23.969898 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:00:23.969898 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:00:23.969898 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:00:23.969898 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:00:23.969898 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:00:23.969898 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:00:23.969898 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:00:23.969898 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:00:23.969898 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:00:23.972000 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:00:23.972000 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:00:23.972000 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:00:23.972000 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:00:23.972000 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:00:23.972000 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:00:23.972000 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:00:23.972000 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:00:23.972000 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:00:23.972000 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:00:23.972000 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:00:23.972000 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:00:23.972000 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:00:23.972000 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:00:23.972000 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:00:23.972000 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:00:23.973876 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:00:23.973876 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:00:23.973876 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:00:23.973876 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:00:23.973876 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:00:23.973876 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:00:23.973876 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:00:23.973876 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:00:23.973876 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:00:23.973876 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:00:23.973876 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 25 03:00:23.973876 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:00:23.973876 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 118/242] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Aug 25 03:00:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:00:23.973876 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dcb201-4622-4e38-8dcb-4b4ba21dac03 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:24.048218 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5db185-a9ca-4ddf-9f34-51142adc4571 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:24.074702 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:24.084592 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dcb201-4622-4e38-8dcb-4b4ba21dac03 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:24.090406 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787f53f6-55cf-461c-b312-9e4892473709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:24.090406 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787f53f6-55cf-461c-b312-9e4892473709 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:24.090406 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787f53f6-55cf-461c-b312-9e4892473709 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:24.090406 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787f53f6-55cf-461c-b312-9e4892473709 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:24.090751 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787f53f6-55cf-461c-b312-9e4892473709 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:24.092020 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787f53f6-55cf-461c-b312-9e4892473709 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:24.098955 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 125/243] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:00:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:24.100449 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787f53f6-55cf-461c-b312-9e4892473709 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:24.103657 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:24.123997 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8da23a8-d949-49d1-a04a-48c05a82d3c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:24.124749 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8da23a8-d949-49d1-a04a-48c05a82d3c7 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:24.125097 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8da23a8-d949-49d1-a04a-48c05a82d3c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:24.125440 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8da23a8-d949-49d1-a04a-48c05a82d3c7 None admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:24.126672 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8da23a8-d949-49d1-a04a-48c05a82d3c7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:24.127191 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8da23a8-d949-49d1-a04a-48c05a82d3c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:24.128927 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8da23a8-d949-49d1-a04a-48c05a82d3c7 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:24.142400 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 119/244] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Aug 25 03:00:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:24.147061 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8da23a8-d949-49d1-a04a-48c05a82d3c7 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:24.214448 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787f53f6-55cf-461c-b312-9e4892473709 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:24.251909 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497c5ec8-faf2-4649-a295-73f31b30f0e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:24.252530 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497c5ec8-faf2-4649-a295-73f31b30f0e0 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:24.252815 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497c5ec8-faf2-4649-a295-73f31b30f0e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:24.253094 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497c5ec8-faf2-4649-a295-73f31b30f0e0 None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:24.253774 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497c5ec8-faf2-4649-a295-73f31b30f0e0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:24.254080 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497c5ec8-faf2-4649-a295-73f31b30f0e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:24.254673 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8da23a8-d949-49d1-a04a-48c05a82d3c7 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:24.255488 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497c5ec8-faf2-4649-a295-73f31b30f0e0 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:24.265067 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 126/245] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Aug 25 03:00:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:24.266025 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497c5ec8-faf2-4649-a295-73f31b30f0e0 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:24.296156 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1ac4a5-66e2-41fb-bdfa-b6acec1663b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:24.296912 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1ac4a5-66e2-41fb-bdfa-b6acec1663b0 None admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:24.297319 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1ac4a5-66e2-41fb-bdfa-b6acec1663b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:24.297591 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1ac4a5-66e2-41fb-bdfa-b6acec1663b0 None admin] PATH_INFO: `/v3/users` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:24.298239 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1ac4a5-66e2-41fb-bdfa-b6acec1663b0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:24.299826 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1ac4a5-66e2-41fb-bdfa-b6acec1663b0 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:24.334168 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:24.373139 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497c5ec8-faf2-4649-a295-73f31b30f0e0 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:24.375814 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 120/246] 10.222.0.58 () {60 vars in 1132 bytes} [Thu Aug 25 03:00:24 2022] POST /identity/v3/users => generated 293 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:00:24.377789 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1ac4a5-66e2-41fb-bdfa-b6acec1663b0 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:24.405489 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:24.425193 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24f9523-9f1d-4737-b311-61499410863c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:24.425868 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24f9523-9f1d-4737-b311-61499410863c None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:24.426156 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24f9523-9f1d-4737-b311-61499410863c None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:24.426488 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24f9523-9f1d-4737-b311-61499410863c None admin] PATH_INFO: `/v3/users/90cfb8b65288490aadfab2795371d6e2` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:24.428102 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24f9523-9f1d-4737-b311-61499410863c None admin] RBAC: Authorizing `identity:get_user(user_id=90cfb8b65288490aadfab2795371d6e2)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:24.430370 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24f9523-9f1d-4737-b311-61499410863c None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:24.433250 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 127/247] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Aug 25 03:00:24 2022] GET /identity/v3/users/90cfb8b65288490aadfab2795371d6e2 => generated 283 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:24.434566 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf0344e-d7ab-45d4-995a-e1b6fe3e0c9c None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:24.434855 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf0344e-d7ab-45d4-995a-e1b6fe3e0c9c None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:24.435133 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf0344e-d7ab-45d4-995a-e1b6fe3e0c9c None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:24.437185 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 128/248] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:00:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:00:24.443145 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf0344e-d7ab-45d4-995a-e1b6fe3e0c9c None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:24.522822 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1ac4a5-66e2-41fb-bdfa-b6acec1663b0 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:24.568155 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf0344e-d7ab-45d4-995a-e1b6fe3e0c9c None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:24.568350 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be62048-f111-4862-b9c6-9bdddaf1846c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:24.568350 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be62048-f111-4862-b9c6-9bdddaf1846c None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:24.568350 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be62048-f111-4862-b9c6-9bdddaf1846c None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:24.568698 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be62048-f111-4862-b9c6-9bdddaf1846c None admin] PATH_INFO: `/v3/users/a644267127e749a9834262937ba37ec4` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:24.590562 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be62048-f111-4862-b9c6-9bdddaf1846c None admin] RBAC: Authorizing `identity:get_user(user_id=a644267127e749a9834262937ba37ec4)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:24.590865 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be62048-f111-4862-b9c6-9bdddaf1846c None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:24.592771 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 121/249] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Aug 25 03:00:24 2022] GET /identity/v3/users/a644267127e749a9834262937ba37ec4 => generated 297 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:24.593752 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97e48a0-2ee3-4af9-909f-65ecbf8f0d1a None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:24.594430 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97e48a0-2ee3-4af9-909f-65ecbf8f0d1a None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:24.594430 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97e48a0-2ee3-4af9-909f-65ecbf8f0d1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:24.609467 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:24.614889 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-d97e48a0-2ee3-4af9-909f-65ecbf8f0d1a None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:00:24.616172 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-d97e48a0-2ee3-4af9-909f-65ecbf8f0d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:24.623777 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d91963-69ce-415e-85fa-e60b8f0bd404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:24.623777 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d91963-69ce-415e-85fa-e60b8f0bd404 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:24.623777 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d91963-69ce-415e-85fa-e60b8f0bd404 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:24.623777 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d91963-69ce-415e-85fa-e60b8f0bd404 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:24.697459 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 122/250] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:00:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 03:00:24.699155 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221245c1-8445-4a9c-9155-66703dfa95f8 None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:24.699262 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221245c1-8445-4a9c-9155-66703dfa95f8 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:24.699504 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221245c1-8445-4a9c-9155-66703dfa95f8 None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:24.700904 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 123/251] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:00:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:00:24.701556 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221245c1-8445-4a9c-9155-66703dfa95f8 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:24.705605 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:24.711437 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d91963-69ce-415e-85fa-e60b8f0bd404 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:24.712979 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d91963-69ce-415e-85fa-e60b8f0bd404 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:24.822729 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 129/252] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Aug 25 03:00:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 25 03:00:24.823985 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7753d63b-aab8-46d9-bf21-849bb8e3e42d None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:24.824274 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7753d63b-aab8-46d9-bf21-849bb8e3e42d None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:24.824547 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7753d63b-aab8-46d9-bf21-849bb8e3e42d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:24.830196 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221245c1-8445-4a9c-9155-66703dfa95f8 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:24.843952 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-7753d63b-aab8-46d9-bf21-849bb8e3e42d None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:00:24.845064 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-7753d63b-aab8-46d9-bf21-849bb8e3e42d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:24.874672 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:24.890947 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3ba8e7-2b61-4668-ae77-74f649539d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:24.891623 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3ba8e7-2b61-4668-ae77-74f649539d2c None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:24.891906 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3ba8e7-2b61-4668-ae77-74f649539d2c None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:24.892180 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3ba8e7-2b61-4668-ae77-74f649539d2c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:24.964605 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 130/253] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:00:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 03:00:24.967259 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7753d63b-aab8-46d9-bf21-849bb8e3e42d None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:25.005514 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3ba8e7-2b61-4668-ae77-74f649539d2c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:25.007230 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3ba8e7-2b61-4668-ae77-74f649539d2c None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:25.010629 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:25.108673 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7753d63b-aab8-46d9-bf21-849bb8e3e42d None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:25.146102 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90c2cd7-3ae0-4c4f-a868-ed14eaef13c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:25.147483 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90c2cd7-3ae0-4c4f-a868-ed14eaef13c2 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:25.147764 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90c2cd7-3ae0-4c4f-a868-ed14eaef13c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:25.148039 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90c2cd7-3ae0-4c4f-a868-ed14eaef13c2 None admin] PATH_INFO: `/v3/projects/54952dee2ab9420d9541e99a1825f5af` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:25.149651 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90c2cd7-3ae0-4c4f-a868-ed14eaef13c2 None admin] RBAC: Authorizing `identity:get_project(project_id=54952dee2ab9420d9541e99a1825f5af)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:25.151353 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90c2cd7-3ae0-4c4f-a868-ed14eaef13c2 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:25.152979 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 131/254] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Aug 25 03:00:24 2022] GET /identity/v3/projects/54952dee2ab9420d9541e99a1825f5af => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:25.156916 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90c2cd7-3ae0-4c4f-a868-ed14eaef13c2 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:25.200915 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 124/255] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Aug 25 03:00:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 499 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 25 03:00:25.202191 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3ba8e7-2b61-4668-ae77-74f649539d2c None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:25.245924 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:25.323405 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90c2cd7-3ae0-4c4f-a868-ed14eaef13c2 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:25.327821 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:25.331808 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3ba8e7-2b61-4668-ae77-74f649539d2c None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:25.366622 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c9f105-ebdd-48f2-8085-e224dae01473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:25.367152 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c9f105-ebdd-48f2-8085-e224dae01473 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:25.367436 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c9f105-ebdd-48f2-8085-e224dae01473 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:25.367709 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c9f105-ebdd-48f2-8085-e224dae01473 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:25.368446 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c9f105-ebdd-48f2-8085-e224dae01473 None admin] RBAC: Adding query filter params (role.id=62f19ea272b447ccaab1576cd9bf4fd9, scope.project.id=54952dee2ab9420d9541e99a1825f5af, user.id=90cfb8b65288490aadfab2795371d6e2) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:25.368976 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c9f105-ebdd-48f2-8085-e224dae01473 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:25.370469 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c9f105-ebdd-48f2-8085-e224dae01473 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:25.371107 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ee04fe-27b2-47c2-9b7d-5f0d3c331c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:25.371944 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ee04fe-27b2-47c2-9b7d-5f0d3c331c78 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:25.372615 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ee04fe-27b2-47c2-9b7d-5f0d3c331c78 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:25.373075 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ee04fe-27b2-47c2-9b7d-5f0d3c331c78 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:25.377090 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 125/256] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Aug 25 03:00:25 2022] GET /identity/v3/role_assignments?user.id=90cfb8b65288490aadfab2795371d6e2&scope.project.id=54952dee2ab9420d9541e99a1825f5af&role.id=62f19ea272b447ccaab1576cd9bf4fd9 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:25.378023 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c9f105-ebdd-48f2-8085-e224dae01473 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:25.388060 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ee04fe-27b2-47c2-9b7d-5f0d3c331c78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:00:25.396193 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-90ee04fe-27b2-47c2-9b7d-5f0d3c331c78 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 25 03:00:25.396193 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:00:25.396193 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:00:25.396193 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:00:25.396193 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:00:25.396193 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:00:25.396193 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:00:25.396193 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:00:25.396193 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:00:25.396193 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:00:25.396193 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:00:25.396193 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:00:25.396193 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:00:25.396193 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:00:25.396193 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:00:25.396193 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:00:25.397148 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:00:25.397148 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:00:25.397148 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:00:25.397148 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:00:25.397148 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:00:25.397148 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:00:25.397148 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:00:25.397148 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:00:25.397148 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:00:25.397148 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:00:25.397148 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:00:25.397148 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:00:25.397148 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:00:25.397148 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:00:25.397148 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:00:25.397148 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:00:25.397901 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:00:25.397901 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:00:25.397901 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:00:25.397901 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:00:25.397901 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:00:25.397901 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:00:25.397901 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:00:25.397901 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:00:25.397901 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:00:25.397901 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:00:25.397901 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 25 03:00:25.397901 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:00:25.399809 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 132/257] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Aug 25 03:00:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:00:25.402300 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ee04fe-27b2-47c2-9b7d-5f0d3c331c78 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:25.497532 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c9f105-ebdd-48f2-8085-e224dae01473 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:25.536325 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaa17f3-47a4-40b6-baa0-9726a304d55f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:25.537411 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaa17f3-47a4-40b6-baa0-9726a304d55f None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:25.537958 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaa17f3-47a4-40b6-baa0-9726a304d55f None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:25.538487 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaa17f3-47a4-40b6-baa0-9726a304d55f None admin] PATH_INFO: `/v3/projects/54952dee2ab9420d9541e99a1825f5af` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:25.539917 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaa17f3-47a4-40b6-baa0-9726a304d55f None admin] RBAC: Authorizing `identity:get_project(project_id=54952dee2ab9420d9541e99a1825f5af)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:25.541548 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaa17f3-47a4-40b6-baa0-9726a304d55f None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:25.543556 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 126/258] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Aug 25 03:00:25 2022] GET /identity/v3/projects/54952dee2ab9420d9541e99a1825f5af => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:25.544528 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaa17f3-47a4-40b6-baa0-9726a304d55f None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:25.580136 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:25.595175 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ee04fe-27b2-47c2-9b7d-5f0d3c331c78 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:25.597952 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:25.640625 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b090f64a-7b0f-4ab8-82b9-df90771ba18f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:25.641809 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b090f64a-7b0f-4ab8-82b9-df90771ba18f None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:25.642094 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b090f64a-7b0f-4ab8-82b9-df90771ba18f None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:25.642423 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b090f64a-7b0f-4ab8-82b9-df90771ba18f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:25.656366 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b090f64a-7b0f-4ab8-82b9-df90771ba18f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:00:25.662772 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-b090f64a-7b0f-4ab8-82b9-df90771ba18f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 25 03:00:25.662772 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:00:25.662772 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:00:25.662772 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:00:25.662772 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:00:25.662772 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:00:25.662772 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:00:25.662772 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:00:25.662772 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:00:25.662772 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:00:25.662772 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:00:25.662772 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:00:25.662772 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:00:25.662772 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:00:25.662772 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:00:25.662772 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:00:25.663648 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:00:25.663648 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:00:25.663648 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:00:25.663648 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:00:25.663648 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:00:25.663648 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:00:25.663648 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:00:25.663648 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:00:25.663648 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:00:25.663648 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:00:25.663648 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:00:25.663648 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:00:25.663648 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:00:25.663648 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:00:25.663648 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:00:25.663648 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:00:25.664454 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:00:25.664454 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:00:25.664454 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:00:25.664454 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:00:25.664454 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:00:25.664454 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:00:25.664454 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:00:25.664454 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:00:25.664454 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:00:25.664454 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:00:25.664454 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 25 03:00:25.664454 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:00:25.665768 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 133/259] 10.222.0.58 () {56 vars in 1114 bytes} [Thu Aug 25 03:00:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:00:25.665914 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b090f64a-7b0f-4ab8-82b9-df90771ba18f None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:25.671906 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaa17f3-47a4-40b6-baa0-9726a304d55f None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:25.712117 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412a44b1-fefe-46cb-9715-ae96f6177894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:25.712730 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412a44b1-fefe-46cb-9715-ae96f6177894 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:25.713067 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412a44b1-fefe-46cb-9715-ae96f6177894 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:25.713342 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412a44b1-fefe-46cb-9715-ae96f6177894 None admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:25.714064 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412a44b1-fefe-46cb-9715-ae96f6177894 None admin] RBAC: Adding query filter params (name=member) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:25.714403 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412a44b1-fefe-46cb-9715-ae96f6177894 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:25.715792 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412a44b1-fefe-46cb-9715-ae96f6177894 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:25.725856 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 127/260] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Aug 25 03:00:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:25.730303 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412a44b1-fefe-46cb-9715-ae96f6177894 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:25.812747 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b090f64a-7b0f-4ab8-82b9-df90771ba18f None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:25.847290 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:25.860594 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412a44b1-fefe-46cb-9715-ae96f6177894 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:25.861449 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17baddbe-ca30-4549-b962-8b656cf68f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:25.862356 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17baddbe-ca30-4549-b962-8b656cf68f9b None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:25.862837 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17baddbe-ca30-4549-b962-8b656cf68f9b None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:25.863362 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17baddbe-ca30-4549-b962-8b656cf68f9b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:25.864346 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17baddbe-ca30-4549-b962-8b656cf68f9b None admin] RBAC: Adding query filter params (role.id=3de9d2fd0b464c399f67fb5240905596, scope.project.id=54952dee2ab9420d9541e99a1825f5af, user.id=a644267127e749a9834262937ba37ec4) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:25.864917 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17baddbe-ca30-4549-b962-8b656cf68f9b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:25.866697 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17baddbe-ca30-4549-b962-8b656cf68f9b None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:25.872801 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 134/261] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Aug 25 03:00:25 2022] GET /identity/v3/role_assignments?user.id=a644267127e749a9834262937ba37ec4&scope.project.id=54952dee2ab9420d9541e99a1825f5af&role.id=3de9d2fd0b464c399f67fb5240905596 => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:25.874227 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17baddbe-ca30-4549-b962-8b656cf68f9b None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:25.897048 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:25.933832 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299bcc86-6b0f-49c1-8e25-6101940cf342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:25.934487 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299bcc86-6b0f-49c1-8e25-6101940cf342 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:25.934771 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299bcc86-6b0f-49c1-8e25-6101940cf342 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:25.935047 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299bcc86-6b0f-49c1-8e25-6101940cf342 None admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:25.936052 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299bcc86-6b0f-49c1-8e25-6101940cf342 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:25.936052 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299bcc86-6b0f-49c1-8e25-6101940cf342 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:25.937536 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299bcc86-6b0f-49c1-8e25-6101940cf342 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:25.947230 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 128/262] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Aug 25 03:00:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:25.953033 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299bcc86-6b0f-49c1-8e25-6101940cf342 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:26.017523 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17baddbe-ca30-4549-b962-8b656cf68f9b None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:26.069508 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8e5b08-6d4e-42b4-ab17-7587019793c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:26.071069 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8e5b08-6d4e-42b4-ab17-7587019793c3 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:26.071358 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8e5b08-6d4e-42b4-ab17-7587019793c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:26.071635 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8e5b08-6d4e-42b4-ab17-7587019793c3 None admin] PATH_INFO: `/v3/users/87b51680f137409c97169ee267d279e5` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:26.073777 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8e5b08-6d4e-42b4-ab17-7587019793c3 None admin] RBAC: Authorizing `identity:get_user(user_id=87b51680f137409c97169ee267d279e5)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:26.075478 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8e5b08-6d4e-42b4-ab17-7587019793c3 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:26.077522 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 135/263] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Aug 25 03:00:25 2022] GET /identity/v3/users/87b51680f137409c97169ee267d279e5 => generated 297 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:26.083286 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8e5b08-6d4e-42b4-ab17-7587019793c3 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:26.120507 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299bcc86-6b0f-49c1-8e25-6101940cf342 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:26.187590 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:26.195391 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9d64e4-64c3-43e4-be76-9586ba124c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:26.195920 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9d64e4-64c3-43e4-be76-9586ba124c36 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:26.196202 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9d64e4-64c3-43e4-be76-9586ba124c36 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:26.196477 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9d64e4-64c3-43e4-be76-9586ba124c36 None admin] PATH_INFO: `/v3/users/90cfb8b65288490aadfab2795371d6e2` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:26.203056 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9d64e4-64c3-43e4-be76-9586ba124c36 None admin] RBAC: Authorizing `identity:get_user(user_id=90cfb8b65288490aadfab2795371d6e2)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:26.204536 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9d64e4-64c3-43e4-be76-9586ba124c36 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:26.206361 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 129/264] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Aug 25 03:00:25 2022] GET /identity/v3/users/90cfb8b65288490aadfab2795371d6e2 => generated 283 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:00:26.209636 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8e5b08-6d4e-42b4-ab17-7587019793c3 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:26.211773 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9d64e4-64c3-43e4-be76-9586ba124c36 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:26.252984 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4e6ae0-1523-4c75-9a5c-cb878e800eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:26.254553 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4e6ae0-1523-4c75-9a5c-cb878e800eab None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:26.255288 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4e6ae0-1523-4c75-9a5c-cb878e800eab None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:26.255767 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4e6ae0-1523-4c75-9a5c-cb878e800eab None admin] PATH_INFO: `/v3/projects/54952dee2ab9420d9541e99a1825f5af` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:26.257117 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4e6ae0-1523-4c75-9a5c-cb878e800eab None admin] RBAC: Authorizing `identity:get_project(project_id=54952dee2ab9420d9541e99a1825f5af)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:26.258819 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4e6ae0-1523-4c75-9a5c-cb878e800eab None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:26.261106 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 136/265] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Aug 25 03:00:26 2022] GET /identity/v3/projects/54952dee2ab9420d9541e99a1825f5af => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:26.267848 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4e6ae0-1523-4c75-9a5c-cb878e800eab None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:26.273176 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:26.393594 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4e6ae0-1523-4c75-9a5c-cb878e800eab None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:26.438303 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0bb217-3dbd-4691-8eb8-092c32f9eda9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:26.439828 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0bb217-3dbd-4691-8eb8-092c32f9eda9 None admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:26.440518 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0bb217-3dbd-4691-8eb8-092c32f9eda9 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:26.441204 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0bb217-3dbd-4691-8eb8-092c32f9eda9 None admin] PATH_INFO: `/v3/projects/54952dee2ab9420d9541e99a1825f5af/users/87b51680f137409c97169ee267d279e5/roles/3255f77192524a16bc405173106871fc` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:26.443853 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0bb217-3dbd-4691-8eb8-092c32f9eda9 None admin] RBAC: Authorizing `identity:create_grant(project_id=54952dee2ab9420d9541e99a1825f5af, user_id=87b51680f137409c97169ee267d279e5, role_id=3255f77192524a16bc405173106871fc)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:26.446181 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0bb217-3dbd-4691-8eb8-092c32f9eda9 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:26.468530 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9d64e4-64c3-43e4-be76-9586ba124c36 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:26.503709 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198411fd-15c5-4bf5-96ba-a04a155c8a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:26.504322 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198411fd-15c5-4bf5-96ba-a04a155c8a3a None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:26.504600 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198411fd-15c5-4bf5-96ba-a04a155c8a3a None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:26.504871 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198411fd-15c5-4bf5-96ba-a04a155c8a3a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:26.530309 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 137/266] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Aug 25 03:00:26 2022] PUT /identity/v3/projects/54952dee2ab9420d9541e99a1825f5af/users/87b51680f137409c97169ee267d279e5/roles/3255f77192524a16bc405173106871fc => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:00:26.582004 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198411fd-15c5-4bf5-96ba-a04a155c8a3a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:26.583574 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198411fd-15c5-4bf5-96ba-a04a155c8a3a None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:26.611037 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:26.689515 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 130/267] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Aug 25 03:00:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 25 03:00:26.695154 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0bb217-3dbd-4691-8eb8-092c32f9eda9 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:26.709375 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:26.813415 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0bb217-3dbd-4691-8eb8-092c32f9eda9 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:26.850781 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d207305-9b4d-4285-a3a0-59413a7c6547 None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:26.851128 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d207305-9b4d-4285-a3a0-59413a7c6547 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:26.851334 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d207305-9b4d-4285-a3a0-59413a7c6547 None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:26.852837 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5df66f-968e-4c98-9eef-a09397095148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:26.852965 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 131/268] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:00:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 25 03:00:26.853249 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5df66f-968e-4c98-9eef-a09397095148 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:26.853530 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5df66f-968e-4c98-9eef-a09397095148 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:26.853858 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5df66f-968e-4c98-9eef-a09397095148 None admin] PATH_INFO: `/v3/projects/54952dee2ab9420d9541e99a1825f5af` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:26.855031 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5df66f-968e-4c98-9eef-a09397095148 None admin] RBAC: Authorizing `identity:get_project(project_id=54952dee2ab9420d9541e99a1825f5af)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:26.856485 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5df66f-968e-4c98-9eef-a09397095148 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:26.857867 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2ff523-37c9-432c-a083-ca340893cae4 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:26.858764 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2ff523-37c9-432c-a083-ca340893cae4 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:26.858990 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 138/269] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Aug 25 03:00:26 2022] GET /identity/v3/projects/54952dee2ab9420d9541e99a1825f5af => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:26.859079 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2ff523-37c9-432c-a083-ca340893cae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:26.867086 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5df66f-968e-4c98-9eef-a09397095148 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:26.885430 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-7a2ff523-37c9-432c-a083-ca340893cae4 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:00:26.886689 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-7a2ff523-37c9-432c-a083-ca340893cae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:26.991596 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:27.015885 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5df66f-968e-4c98-9eef-a09397095148 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:27.016711 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 132/270] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:00:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 03:00:27.066081 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3387da5-7541-446c-8bb0-504a22c53fe9 None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:27.066406 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3387da5-7541-446c-8bb0-504a22c53fe9 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:27.066685 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3387da5-7541-446c-8bb0-504a22c53fe9 None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:27.068180 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 133/271] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:00:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 25 03:00:27.071108 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf40aee1-494b-4eef-aa5f-0c1a50ed6a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:27.071977 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf40aee1-494b-4eef-aa5f-0c1a50ed6a77 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:27.072432 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf40aee1-494b-4eef-aa5f-0c1a50ed6a77 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:27.072898 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf40aee1-494b-4eef-aa5f-0c1a50ed6a77 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:27.073801 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf40aee1-494b-4eef-aa5f-0c1a50ed6a77 None admin] RBAC: Adding query filter params (role.id=7886cc7e333a42d08085932494a3ea6b, scope.project.id=54952dee2ab9420d9541e99a1825f5af, user.id=90cfb8b65288490aadfab2795371d6e2) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:27.074316 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf40aee1-494b-4eef-aa5f-0c1a50ed6a77 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:27.075925 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf40aee1-494b-4eef-aa5f-0c1a50ed6a77 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:27.084018 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62e7bf4-d2d7-456d-8ad1-b74cedad486c None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:27.084313 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62e7bf4-d2d7-456d-8ad1-b74cedad486c None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:27.084593 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62e7bf4-d2d7-456d-8ad1-b74cedad486c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:27.087918 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 139/272] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Aug 25 03:00:26 2022] GET /identity/v3/role_assignments?user.id=90cfb8b65288490aadfab2795371d6e2&scope.project.id=54952dee2ab9420d9541e99a1825f5af&role.id=7886cc7e333a42d08085932494a3ea6b => generated 608 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:00:27.108711 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-a62e7bf4-d2d7-456d-8ad1-b74cedad486c None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:00:27.110218 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-a62e7bf4-d2d7-456d-8ad1-b74cedad486c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:27.164307 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:27.185815 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 134/273] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:00:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 03:00:27.300500 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf40aee1-494b-4eef-aa5f-0c1a50ed6a77 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:27.410234 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf40aee1-494b-4eef-aa5f-0c1a50ed6a77 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:27.444314 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531c6170-ae8c-4dec-8a25-d0852394712b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:27.445239 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531c6170-ae8c-4dec-8a25-d0852394712b None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:27.445695 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531c6170-ae8c-4dec-8a25-d0852394712b None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:27.446140 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531c6170-ae8c-4dec-8a25-d0852394712b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:27.457432 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531c6170-ae8c-4dec-8a25-d0852394712b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:00:27.462915 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-531c6170-ae8c-4dec-8a25-d0852394712b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 25 03:00:27.462915 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:00:27.462915 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:00:27.462915 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:00:27.462915 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:00:27.462915 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:00:27.462915 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:00:27.462915 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:00:27.462915 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:00:27.462915 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:00:27.462915 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:00:27.462915 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:00:27.462915 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:00:27.462915 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:00:27.462915 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:00:27.462915 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:00:27.463782 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:00:27.463782 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:00:27.463782 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:00:27.463782 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:00:27.463782 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:00:27.463782 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:00:27.463782 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:00:27.463782 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:00:27.463782 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:00:27.463782 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:00:27.463782 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:00:27.463782 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:00:27.463782 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:00:27.463782 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:00:27.463782 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:00:27.463782 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:00:27.464560 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:00:27.464560 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:00:27.464560 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:00:27.464560 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:00:27.464560 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:00:27.464560 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:00:27.464560 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:00:27.464560 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:00:27.464560 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:00:27.464560 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:00:27.464560 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 25 03:00:27.464560 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:00:27.466722 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 140/274] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Aug 25 03:00:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:00:27.472264 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62e7bf4-d2d7-456d-8ad1-b74cedad486c None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:27.549366 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531c6170-ae8c-4dec-8a25-d0852394712b None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:27.552696 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:27.582696 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62e7bf4-d2d7-456d-8ad1-b74cedad486c None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:27.621176 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8dca38-dc25-4467-a808-c59c7de33df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:27.621793 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8dca38-dc25-4467-a808-c59c7de33df6 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:27.622072 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8dca38-dc25-4467-a808-c59c7de33df6 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:27.622373 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8dca38-dc25-4467-a808-c59c7de33df6 None admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:27.623048 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8dca38-dc25-4467-a808-c59c7de33df6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:27.623348 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8dca38-dc25-4467-a808-c59c7de33df6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:27.625311 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8dca38-dc25-4467-a808-c59c7de33df6 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:27.638871 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 135/275] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Aug 25 03:00:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:00:27.645429 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8dca38-dc25-4467-a808-c59c7de33df6 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:27.673665 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531c6170-ae8c-4dec-8a25-d0852394712b None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:27.693388 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:27.713020 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489ab093-18e9-413d-9ee0-fa9f3f4d44af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:27.715291 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489ab093-18e9-413d-9ee0-fa9f3f4d44af None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:27.715291 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489ab093-18e9-413d-9ee0-fa9f3f4d44af None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:27.715291 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489ab093-18e9-413d-9ee0-fa9f3f4d44af None admin] PATH_INFO: `/v3/domains/default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:27.716067 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489ab093-18e9-413d-9ee0-fa9f3f4d44af None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:27.717739 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489ab093-18e9-413d-9ee0-fa9f3f4d44af None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:27.723550 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 141/276] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:00:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:00:27.725612 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489ab093-18e9-413d-9ee0-fa9f3f4d44af None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:27.753026 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8dca38-dc25-4467-a808-c59c7de33df6 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:27.787347 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7694a6c-b2f5-4fc3-b196-d2d3448e239b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:27.787982 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7694a6c-b2f5-4fc3-b196-d2d3448e239b None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:27.788270 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7694a6c-b2f5-4fc3-b196-d2d3448e239b None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:27.788535 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7694a6c-b2f5-4fc3-b196-d2d3448e239b None admin] PATH_INFO: `/v3/users/01aa00c8ce8c45fc9946621875d1d064` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:27.800668 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:27.803430 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7694a6c-b2f5-4fc3-b196-d2d3448e239b None admin] RBAC: Authorizing `identity:get_user(user_id=01aa00c8ce8c45fc9946621875d1d064)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:27.804862 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7694a6c-b2f5-4fc3-b196-d2d3448e239b None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:27.806784 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 136/277] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Aug 25 03:00:27 2022] GET /identity/v3/users/01aa00c8ce8c45fc9946621875d1d064 => generated 293 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:27.812983 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7694a6c-b2f5-4fc3-b196-d2d3448e239b None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:27.834747 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489ab093-18e9-413d-9ee0-fa9f3f4d44af None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:27.870925 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a9763c-727d-4a2c-bac4-75df85ff5daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:27.871869 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a9763c-727d-4a2c-bac4-75df85ff5daf None admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:27.872358 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a9763c-727d-4a2c-bac4-75df85ff5daf None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:27.872809 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a9763c-727d-4a2c-bac4-75df85ff5daf None admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:27.873645 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a9763c-727d-4a2c-bac4-75df85ff5daf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:27.876985 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a9763c-727d-4a2c-bac4-75df85ff5daf None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:27.921200 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7694a6c-b2f5-4fc3-b196-d2d3448e239b None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:27.923056 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:27.955824 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7125f662-6a3e-4406-9fac-8ec5bedd6052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:27.956539 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7125f662-6a3e-4406-9fac-8ec5bedd6052 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:27.956825 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7125f662-6a3e-4406-9fac-8ec5bedd6052 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:27.957097 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7125f662-6a3e-4406-9fac-8ec5bedd6052 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:27.957843 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7125f662-6a3e-4406-9fac-8ec5bedd6052 None admin] RBAC: Adding query filter params (role.id=3255f77192524a16bc405173106871fc, user.id=01aa00c8ce8c45fc9946621875d1d064, scope.system=all) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:27.958154 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7125f662-6a3e-4406-9fac-8ec5bedd6052 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:27.959677 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7125f662-6a3e-4406-9fac-8ec5bedd6052 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:27.972972 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 137/278] 10.222.0.58 () {56 vars in 1308 bytes} [Thu Aug 25 03:00:27 2022] GET /identity/v3/role_assignments?user.id=01aa00c8ce8c45fc9946621875d1d064&scope.system=all&role.id=3255f77192524a16bc405173106871fc => generated 226 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:00:27.978581 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 142/279] 10.222.0.58 () {60 vars in 1132 bytes} [Thu Aug 25 03:00:27 2022] POST /identity/v3/users => generated 293 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:00:28.176080 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a84a13-5908-48e6-84ca-00f5c1d52570 None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:28.176526 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a84a13-5908-48e6-84ca-00f5c1d52570 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:28.176886 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a84a13-5908-48e6-84ca-00f5c1d52570 None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:28.178483 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 138/280] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:00:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:00:28.183663 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bb6037-b3fa-43f6-b722-954923de9bd6 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:28.184107 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bb6037-b3fa-43f6-b722-954923de9bd6 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:28.184474 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bb6037-b3fa-43f6-b722-954923de9bd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:28.206373 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-f5bb6037-b3fa-43f6-b722-954923de9bd6 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:00:28.207682 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-f5bb6037-b3fa-43f6-b722-954923de9bd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:28.279508 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 143/281] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:00:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 03:00:28.561906 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a84a13-5908-48e6-84ca-00f5c1d52570 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:28.691133 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a84a13-5908-48e6-84ca-00f5c1d52570 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:28.727653 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678afb07-2b09-4356-8bec-ba8d1a8b5bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:28.728450 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678afb07-2b09-4356-8bec-ba8d1a8b5bb3 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:28.728730 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678afb07-2b09-4356-8bec-ba8d1a8b5bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:28.729001 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678afb07-2b09-4356-8bec-ba8d1a8b5bb3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:28.739769 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678afb07-2b09-4356-8bec-ba8d1a8b5bb3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:00:28.744590 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-678afb07-2b09-4356-8bec-ba8d1a8b5bb3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 25 03:00:28.744590 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:00:28.744590 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:00:28.744590 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:00:28.744590 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:00:28.744590 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:00:28.744590 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:00:28.744590 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:00:28.744590 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:00:28.744590 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:00:28.744590 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:00:28.744590 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:00:28.744590 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:00:28.744590 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:00:28.744590 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:00:28.744590 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:00:28.745431 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:00:28.745431 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:00:28.745431 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:00:28.745431 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:00:28.745431 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:00:28.745431 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:00:28.745431 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:00:28.745431 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:00:28.745431 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:00:28.745431 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:00:28.745431 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:00:28.745431 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:00:28.745431 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:00:28.745431 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:00:28.745431 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:00:28.745431 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:00:28.746392 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:00:28.746392 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:00:28.746392 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:00:28.746392 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:00:28.746392 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:00:28.746392 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:00:28.746392 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:00:28.746392 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:00:28.746392 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:00:28.746392 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:00:28.746392 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 25 03:00:28.746392 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:00:28.746392 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 139/282] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Aug 25 03:00:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:00:28.753135 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bb6037-b3fa-43f6-b722-954923de9bd6 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:28.814998 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fff5058-a620-4618-9f82-cc7f58a17695 None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:28.815291 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fff5058-a620-4618-9f82-cc7f58a17695 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:28.815599 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fff5058-a620-4618-9f82-cc7f58a17695 None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:28.817196 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 140/283] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:00:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:00:28.823508 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d1ad1c-dd3e-4212-8532-a6125fc0ba4d None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:28.823912 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d1ad1c-dd3e-4212-8532-a6125fc0ba4d None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:28.824379 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d1ad1c-dd3e-4212-8532-a6125fc0ba4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:28.849935 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-23d1ad1c-dd3e-4212-8532-a6125fc0ba4d None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:00:28.854127 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:28.854914 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-23d1ad1c-dd3e-4212-8532-a6125fc0ba4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:28.898242 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:28.939506 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bb6037-b3fa-43f6-b722-954923de9bd6 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:28.985920 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 141/284] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:00:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 25 03:00:28.997483 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60743be4-0e0e-4f1d-bf4b-0721ec39f09c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:28.998114 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60743be4-0e0e-4f1d-bf4b-0721ec39f09c None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:28.998427 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60743be4-0e0e-4f1d-bf4b-0721ec39f09c None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:28.998729 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60743be4-0e0e-4f1d-bf4b-0721ec39f09c None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:28.999415 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60743be4-0e0e-4f1d-bf4b-0721ec39f09c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:28.999719 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60743be4-0e0e-4f1d-bf4b-0721ec39f09c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:29.001113 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60743be4-0e0e-4f1d-bf4b-0721ec39f09c None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:29.013804 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 144/285] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Aug 25 03:00:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:29.020638 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d1ad1c-dd3e-4212-8532-a6125fc0ba4d None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:29.136301 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d1ad1c-dd3e-4212-8532-a6125fc0ba4d None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:29.172298 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d668df-a2a3-4a1f-880e-72c5a8d9acf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:29.173226 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d668df-a2a3-4a1f-880e-72c5a8d9acf9 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:29.173682 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d668df-a2a3-4a1f-880e-72c5a8d9acf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:29.174127 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d668df-a2a3-4a1f-880e-72c5a8d9acf9 None admin] PATH_INFO: `/v3/users/a644267127e749a9834262937ba37ec4` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:29.175433 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d668df-a2a3-4a1f-880e-72c5a8d9acf9 None admin] RBAC: Authorizing `identity:get_user(user_id=a644267127e749a9834262937ba37ec4)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:29.177095 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d668df-a2a3-4a1f-880e-72c5a8d9acf9 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:29.179238 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 142/286] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Aug 25 03:00:29 2022] GET /identity/v3/users/a644267127e749a9834262937ba37ec4 => generated 297 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:29.184443 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60743be4-0e0e-4f1d-bf4b-0721ec39f09c None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:29.334562 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d668df-a2a3-4a1f-880e-72c5a8d9acf9 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:29.353263 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60743be4-0e0e-4f1d-bf4b-0721ec39f09c None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:29.378109 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:29.395438 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:29.443869 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bb0d4e-f52c-4fe2-8914-a8322e929785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:29.444546 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bb0d4e-f52c-4fe2-8914-a8322e929785 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:29.444831 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bb0d4e-f52c-4fe2-8914-a8322e929785 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:29.445105 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bb0d4e-f52c-4fe2-8914-a8322e929785 None admin] PATH_INFO: `/v3/projects/54952dee2ab9420d9541e99a1825f5af` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:29.451099 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bb0d4e-f52c-4fe2-8914-a8322e929785 None admin] RBAC: Authorizing `identity:get_project(project_id=54952dee2ab9420d9541e99a1825f5af)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:29.453808 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bb0d4e-f52c-4fe2-8914-a8322e929785 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:29.455888 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 145/287] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Aug 25 03:00:29 2022] GET /identity/v3/projects/54952dee2ab9420d9541e99a1825f5af => generated 302 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:29.465139 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d668df-a2a3-4a1f-880e-72c5a8d9acf9 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:29.467245 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bb0d4e-f52c-4fe2-8914-a8322e929785 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:29.506458 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4204f4-c33e-40cf-9615-8db9da3a72c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:29.508147 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4204f4-c33e-40cf-9615-8db9da3a72c5 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:29.508430 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4204f4-c33e-40cf-9615-8db9da3a72c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:29.508706 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4204f4-c33e-40cf-9615-8db9da3a72c5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:29.530257 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4204f4-c33e-40cf-9615-8db9da3a72c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:00:29.537611 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-be4204f4-c33e-40cf-9615-8db9da3a72c5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 25 03:00:29.537611 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:00:29.537611 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:00:29.537611 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:00:29.537611 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:00:29.537611 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:00:29.537611 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:00:29.537611 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:00:29.537611 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:00:29.537611 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:00:29.537611 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:00:29.537611 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:00:29.537611 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:00:29.537611 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:00:29.537611 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:00:29.537611 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:00:29.538538 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:00:29.538538 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:00:29.538538 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:00:29.538538 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:00:29.538538 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:00:29.538538 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:00:29.538538 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:00:29.538538 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:00:29.538538 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:00:29.538538 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:00:29.538538 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:00:29.538538 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:00:29.538538 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:00:29.538538 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:00:29.538538 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:00:29.538538 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:00:29.539756 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:00:29.539756 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:00:29.539756 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:00:29.539756 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:00:29.539756 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:00:29.539756 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:00:29.539756 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:00:29.539756 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:00:29.539756 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:00:29.539756 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:00:29.539756 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 25 03:00:29.539756 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:00:29.539756 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 143/288] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Aug 25 03:00:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 03:00:29.551083 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4204f4-c33e-40cf-9615-8db9da3a72c5 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:29.664609 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bb0d4e-f52c-4fe2-8914-a8322e929785 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:29.703053 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:29.728206 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4204f4-c33e-40cf-9615-8db9da3a72c5 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:29.730242 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a7ae9b-1130-41e1-bdd7-04eaf76e3c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:29.730940 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a7ae9b-1130-41e1-bdd7-04eaf76e3c11 None admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:29.731221 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a7ae9b-1130-41e1-bdd7-04eaf76e3c11 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:29.731497 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a7ae9b-1130-41e1-bdd7-04eaf76e3c11 None admin] PATH_INFO: `/v3/projects/54952dee2ab9420d9541e99a1825f5af/users/a644267127e749a9834262937ba37ec4/roles/3de9d2fd0b464c399f67fb5240905596` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:29.734224 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a7ae9b-1130-41e1-bdd7-04eaf76e3c11 None admin] RBAC: Authorizing `identity:create_grant(project_id=54952dee2ab9420d9541e99a1825f5af, user_id=a644267127e749a9834262937ba37ec4, role_id=3de9d2fd0b464c399f67fb5240905596)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:29.735694 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a7ae9b-1130-41e1-bdd7-04eaf76e3c11 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:29.772208 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a7876c-b0a7-4c8b-ab8a-99e9cd2fb723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:29.772832 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a7876c-b0a7-4c8b-ab8a-99e9cd2fb723 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:29.773113 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a7876c-b0a7-4c8b-ab8a-99e9cd2fb723 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:29.773391 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a7876c-b0a7-4c8b-ab8a-99e9cd2fb723 None admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:29.774135 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a7876c-b0a7-4c8b-ab8a-99e9cd2fb723 None admin] RBAC: Adding query filter params (name=member) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:29.774471 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a7876c-b0a7-4c8b-ab8a-99e9cd2fb723 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:29.775949 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a7876c-b0a7-4c8b-ab8a-99e9cd2fb723 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:29.785829 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 144/289] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Aug 25 03:00:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:29.791060 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a7876c-b0a7-4c8b-ab8a-99e9cd2fb723 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:29.824914 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 146/290] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Aug 25 03:00:29 2022] PUT /identity/v3/projects/54952dee2ab9420d9541e99a1825f5af/users/a644267127e749a9834262937ba37ec4/roles/3de9d2fd0b464c399f67fb5240905596 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:00:29.918523 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a7876c-b0a7-4c8b-ab8a-99e9cd2fb723 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:29.955644 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d67a8ef-b965-4fb7-b2fe-d259fa19b355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:29.956379 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d67a8ef-b965-4fb7-b2fe-d259fa19b355 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:29.956775 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d67a8ef-b965-4fb7-b2fe-d259fa19b355 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:29.957055 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d67a8ef-b965-4fb7-b2fe-d259fa19b355 None admin] PATH_INFO: `/v3/users/87b51680f137409c97169ee267d279e5` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:29.958310 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d67a8ef-b965-4fb7-b2fe-d259fa19b355 None admin] RBAC: Authorizing `identity:get_user(user_id=87b51680f137409c97169ee267d279e5)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:29.959825 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d67a8ef-b965-4fb7-b2fe-d259fa19b355 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:29.961642 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 145/291] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Aug 25 03:00:29 2022] GET /identity/v3/users/87b51680f137409c97169ee267d279e5 => generated 297 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:00:29.967537 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a7ae9b-1130-41e1-bdd7-04eaf76e3c11 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:30.023074 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:30.090232 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a7ae9b-1130-41e1-bdd7-04eaf76e3c11 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:30.129374 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe554b3-4c58-4499-9a6e-e03e3b81a89e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:30.130055 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe554b3-4c58-4499-9a6e-e03e3b81a89e None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:30.130370 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe554b3-4c58-4499-9a6e-e03e3b81a89e None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:30.130648 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe554b3-4c58-4499-9a6e-e03e3b81a89e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:30.200337 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e954e2b6-52cc-4f30-ac00-30eaa0d4182b None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:30.200520 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e954e2b6-52cc-4f30-ac00-30eaa0d4182b None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:30.200796 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e954e2b6-52cc-4f30-ac00-30eaa0d4182b None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:30.203975 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 146/292] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:00:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:00:30.209406 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe554b3-4c58-4499-9a6e-e03e3b81a89e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:30.211158 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe554b3-4c58-4499-9a6e-e03e3b81a89e None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:30.211518 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bf66ef-98e5-44fd-b47e-6c9d6f4ed571 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:30.211809 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bf66ef-98e5-44fd-b47e-6c9d6f4ed571 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:30.212086 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bf66ef-98e5-44fd-b47e-6c9d6f4ed571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:30.237537 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-97bf66ef-98e5-44fd-b47e-6c9d6f4ed571 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:00:30.239250 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-97bf66ef-98e5-44fd-b47e-6c9d6f4ed571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:30.254665 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:30.263221 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:30.327247 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 147/293] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:00:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 25 03:00:30.359203 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 147/294] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Aug 25 03:00:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 25 03:00:30.364972 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bf66ef-98e5-44fd-b47e-6c9d6f4ed571 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:30.488514 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bf66ef-98e5-44fd-b47e-6c9d6f4ed571 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:30.529874 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665298ee-d8c6-481a-b48c-c7a27fd0abc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:30.531062 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665298ee-d8c6-481a-b48c-c7a27fd0abc6 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:30.531340 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665298ee-d8c6-481a-b48c-c7a27fd0abc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:30.531606 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665298ee-d8c6-481a-b48c-c7a27fd0abc6 None admin] PATH_INFO: `/v3/projects/54952dee2ab9420d9541e99a1825f5af` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:30.532738 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665298ee-d8c6-481a-b48c-c7a27fd0abc6 None admin] RBAC: Authorizing `identity:get_project(project_id=54952dee2ab9420d9541e99a1825f5af)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:30.534288 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665298ee-d8c6-481a-b48c-c7a27fd0abc6 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:30.536211 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 148/295] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Aug 25 03:00:30 2022] GET /identity/v3/projects/54952dee2ab9420d9541e99a1825f5af => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:30.541758 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe554b3-4c58-4499-9a6e-e03e3b81a89e None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:30.623089 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665298ee-d8c6-481a-b48c-c7a27fd0abc6 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:30.643177 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:30.687599 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe554b3-4c58-4499-9a6e-e03e3b81a89e None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:30.724728 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99958eff-d30d-4587-8f5f-6420d912bf10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:30.725346 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99958eff-d30d-4587-8f5f-6420d912bf10 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:30.725674 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99958eff-d30d-4587-8f5f-6420d912bf10 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:30.726148 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99958eff-d30d-4587-8f5f-6420d912bf10 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:30.726747 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99958eff-d30d-4587-8f5f-6420d912bf10 None admin] RBAC: Adding query filter params (role.id=3255f77192524a16bc405173106871fc, scope.project.id=54952dee2ab9420d9541e99a1825f5af, user.id=87b51680f137409c97169ee267d279e5) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:30.727051 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99958eff-d30d-4587-8f5f-6420d912bf10 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:30.728499 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99958eff-d30d-4587-8f5f-6420d912bf10 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:30.742390 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 148/296] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Aug 25 03:00:30 2022] GET /identity/v3/role_assignments?user.id=87b51680f137409c97169ee267d279e5&scope.project.id=54952dee2ab9420d9541e99a1825f5af&role.id=3255f77192524a16bc405173106871fc => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:30.742390 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a30977-8826-4359-aa27-6f506ddc1551 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:30.742390 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a30977-8826-4359-aa27-6f506ddc1551 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:30.742390 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a30977-8826-4359-aa27-6f506ddc1551 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:30.742390 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 149/297] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:00:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:00:30.744682 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddea4c0-2928-4afc-adc2-56c9dacb9c65 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:30.744977 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddea4c0-2928-4afc-adc2-56c9dacb9c65 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:30.745258 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddea4c0-2928-4afc-adc2-56c9dacb9c65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:30.756393 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665298ee-d8c6-481a-b48c-c7a27fd0abc6 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:30.769003 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-cddea4c0-2928-4afc-adc2-56c9dacb9c65 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:00:30.771183 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-cddea4c0-2928-4afc-adc2-56c9dacb9c65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:30.776519 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:30.797048 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f7d72d-4da4-48fb-9252-01e1ff1e58f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:30.797712 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f7d72d-4da4-48fb-9252-01e1ff1e58f2 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:30.798049 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f7d72d-4da4-48fb-9252-01e1ff1e58f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:30.798355 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f7d72d-4da4-48fb-9252-01e1ff1e58f2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:30.810653 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f7d72d-4da4-48fb-9252-01e1ff1e58f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:00:30.815917 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-08f7d72d-4da4-48fb-9252-01e1ff1e58f2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 25 03:00:30.815917 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:00:30.815917 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:00:30.815917 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:00:30.815917 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:00:30.815917 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:00:30.815917 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:00:30.815917 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:00:30.815917 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:00:30.815917 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:00:30.815917 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:00:30.815917 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:00:30.815917 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:00:30.815917 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:00:30.815917 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:00:30.815917 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:00:30.816821 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:00:30.816821 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:00:30.816821 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:00:30.816821 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:00:30.816821 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:00:30.816821 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:00:30.816821 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:00:30.816821 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:00:30.816821 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:00:30.816821 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:00:30.816821 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:00:30.816821 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:00:30.816821 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:00:30.816821 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:00:30.816821 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:00:30.816821 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:00:30.817641 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:00:30.817641 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:00:30.817641 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:00:30.817641 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:00:30.817641 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:00:30.817641 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:00:30.817641 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:00:30.817641 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:00:30.817641 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:00:30.817641 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:00:30.817641 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 25 03:00:30.817641 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:00:30.818234 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 149/298] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Aug 25 03:00:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:00:30.822214 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2a3422-3884-420b-bec7-adb408c32599 None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:30.822214 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2a3422-3884-420b-bec7-adb408c32599 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:30.822214 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2a3422-3884-420b-bec7-adb408c32599 None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:30.822214 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 150/299] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:00:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:00:30.825875 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2a3422-3884-420b-bec7-adb408c32599 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:30.874392 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 150/300] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:00:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 25 03:00:30.876047 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45190fe3-1fba-493a-ac40-6e1156296a27 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:30.876458 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45190fe3-1fba-493a-ac40-6e1156296a27 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:30.876836 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45190fe3-1fba-493a-ac40-6e1156296a27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:30.910501 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:30.911596 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-45190fe3-1fba-493a-ac40-6e1156296a27 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:00:30.912779 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-45190fe3-1fba-493a-ac40-6e1156296a27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:30.943020 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2a3422-3884-420b-bec7-adb408c32599 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:30.979487 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8fc10d-afd6-44c4-86af-8970691e9fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:30.980279 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8fc10d-afd6-44c4-86af-8970691e9fbd None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:30.980481 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8fc10d-afd6-44c4-86af-8970691e9fbd None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:30.980759 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8fc10d-afd6-44c4-86af-8970691e9fbd None admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:30.981434 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8fc10d-afd6-44c4-86af-8970691e9fbd None admin] RBAC: Adding query filter params (name=member) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:30.981737 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8fc10d-afd6-44c4-86af-8970691e9fbd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:30.983280 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8fc10d-afd6-44c4-86af-8970691e9fbd None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:30.992301 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 151/301] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Aug 25 03:00:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:00:30.998616 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8fc10d-afd6-44c4-86af-8970691e9fbd None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:31.013020 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 151/302] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:00:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 03:00:31.119472 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8fc10d-afd6-44c4-86af-8970691e9fbd None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:31.156055 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91b45a1-96a3-4d92-aa94-5bb2ed93bdbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:31.156973 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91b45a1-96a3-4d92-aa94-5bb2ed93bdbe None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:31.157433 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91b45a1-96a3-4d92-aa94-5bb2ed93bdbe None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:31.157885 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91b45a1-96a3-4d92-aa94-5bb2ed93bdbe None admin] PATH_INFO: `/v3/users/01aa00c8ce8c45fc9946621875d1d064` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:31.159648 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91b45a1-96a3-4d92-aa94-5bb2ed93bdbe None admin] RBAC: Authorizing `identity:get_user(user_id=01aa00c8ce8c45fc9946621875d1d064)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:31.161337 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91b45a1-96a3-4d92-aa94-5bb2ed93bdbe None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:31.163453 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 152/303] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Aug 25 03:00:30 2022] GET /identity/v3/users/01aa00c8ce8c45fc9946621875d1d064 => generated 293 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:31.169309 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45190fe3-1fba-493a-ac40-6e1156296a27 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:31.197906 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91b45a1-96a3-4d92-aa94-5bb2ed93bdbe None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:31.200793 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:31.247545 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:31.281011 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45190fe3-1fba-493a-ac40-6e1156296a27 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:31.322013 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91b45a1-96a3-4d92-aa94-5bb2ed93bdbe None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:31.326759 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945e7a71-9258-424b-9eb8-0ab3bc60ea96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:31.328513 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945e7a71-9258-424b-9eb8-0ab3bc60ea96 None admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:31.328975 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945e7a71-9258-424b-9eb8-0ab3bc60ea96 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:31.329428 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945e7a71-9258-424b-9eb8-0ab3bc60ea96 None admin] PATH_INFO: `/v3/system/users/01aa00c8ce8c45fc9946621875d1d064/roles/3255f77192524a16bc405173106871fc` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:31.333059 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945e7a71-9258-424b-9eb8-0ab3bc60ea96 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=01aa00c8ce8c45fc9946621875d1d064, role_id=3255f77192524a16bc405173106871fc)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:31.334481 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945e7a71-9258-424b-9eb8-0ab3bc60ea96 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:31.362952 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8c669d-dd6d-486f-ac6c-f043e5cc28d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:31.363751 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8c669d-dd6d-486f-ac6c-f043e5cc28d7 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:31.364131 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8c669d-dd6d-486f-ac6c-f043e5cc28d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:31.364481 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8c669d-dd6d-486f-ac6c-f043e5cc28d7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:31.375947 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8c669d-dd6d-486f-ac6c-f043e5cc28d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:00:31.381327 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-ac8c669d-dd6d-486f-ac6c-f043e5cc28d7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 25 03:00:31.381327 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:00:31.381327 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:00:31.381327 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:00:31.381327 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:00:31.381327 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:00:31.381327 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:00:31.381327 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:00:31.381327 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:00:31.381327 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:00:31.381327 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:00:31.381327 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:00:31.381327 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:00:31.381327 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:00:31.381327 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:00:31.381327 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:00:31.382215 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:00:31.382215 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:00:31.382215 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:00:31.382215 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:00:31.382215 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:00:31.382215 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:00:31.382215 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:00:31.382215 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:00:31.382215 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:00:31.382215 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:00:31.382215 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:00:31.382215 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:00:31.382215 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:00:31.382215 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:00:31.382215 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:00:31.382215 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:00:31.383045 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:00:31.383045 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:00:31.383045 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:00:31.383045 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:00:31.383045 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:00:31.383045 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:00:31.383045 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:00:31.383045 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:00:31.383045 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:00:31.383045 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:00:31.383045 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 25 03:00:31.383045 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:00:31.383045 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 153/304] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Aug 25 03:00:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 03:00:31.383669 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8c669d-dd6d-486f-ac6c-f043e5cc28d7 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:31.411403 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 152/305] 10.222.0.58 () {58 vars in 1334 bytes} [Thu Aug 25 03:00:31 2022] PUT /identity/v3/system/users/01aa00c8ce8c45fc9946621875d1d064/roles/3255f77192524a16bc405173106871fc => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:00:31.412658 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945e7a71-9258-424b-9eb8-0ab3bc60ea96 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:31.463689 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:31.517161 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8c669d-dd6d-486f-ac6c-f043e5cc28d7 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:31.553360 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945e7a71-9258-424b-9eb8-0ab3bc60ea96 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:31.562837 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa1b528-a3ee-45a5-83b8-07df87dd8437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:31.563691 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa1b528-a3ee-45a5-83b8-07df87dd8437 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:31.564741 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa1b528-a3ee-45a5-83b8-07df87dd8437 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:31.565161 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa1b528-a3ee-45a5-83b8-07df87dd8437 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:31.566508 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa1b528-a3ee-45a5-83b8-07df87dd8437 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:31.567965 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa1b528-a3ee-45a5-83b8-07df87dd8437 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:31.570324 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 154/306] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:00:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:31.576035 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa1b528-a3ee-45a5-83b8-07df87dd8437 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:31.603026 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cccc8a5-4a34-4e8e-8075-b3a9a40b57f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:31.604502 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cccc8a5-4a34-4e8e-8075-b3a9a40b57f0 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:31.604783 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cccc8a5-4a34-4e8e-8075-b3a9a40b57f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:31.605058 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cccc8a5-4a34-4e8e-8075-b3a9a40b57f0 None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:31.605738 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cccc8a5-4a34-4e8e-8075-b3a9a40b57f0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:31.606044 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cccc8a5-4a34-4e8e-8075-b3a9a40b57f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:31.607517 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cccc8a5-4a34-4e8e-8075-b3a9a40b57f0 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:31.609194 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:31.617137 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 153/307] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Aug 25 03:00:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:31.623361 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cccc8a5-4a34-4e8e-8075-b3a9a40b57f0 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:31.727393 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa1b528-a3ee-45a5-83b8-07df87dd8437 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:31.748052 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cccc8a5-4a34-4e8e-8075-b3a9a40b57f0 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:31.770884 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:31.779312 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265a1e32-5397-445a-96ef-b6d70af2a7f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:31.780050 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265a1e32-5397-445a-96ef-b6d70af2a7f1 None admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:31.780771 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265a1e32-5397-445a-96ef-b6d70af2a7f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:31.781061 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265a1e32-5397-445a-96ef-b6d70af2a7f1 None admin] PATH_INFO: `/v3/groups` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:31.781854 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265a1e32-5397-445a-96ef-b6d70af2a7f1 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:31.783749 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265a1e32-5397-445a-96ef-b6d70af2a7f1 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:31.789651 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcbef8b-9fc9-4889-aaf0-1f26a4fa20fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:31.790583 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcbef8b-9fc9-4889-aaf0-1f26a4fa20fd None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:31.791067 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcbef8b-9fc9-4889-aaf0-1f26a4fa20fd None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:31.791653 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcbef8b-9fc9-4889-aaf0-1f26a4fa20fd None admin] PATH_INFO: `/v3/users/d0ab073dde7942f6a951dce04d3822d2` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:31.807281 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcbef8b-9fc9-4889-aaf0-1f26a4fa20fd None admin] RBAC: Authorizing `identity:get_user(user_id=d0ab073dde7942f6a951dce04d3822d2)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:31.808792 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcbef8b-9fc9-4889-aaf0-1f26a4fa20fd None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:31.810763 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 154/308] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Aug 25 03:00:31 2022] GET /identity/v3/users/d0ab073dde7942f6a951dce04d3822d2 => generated 293 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:31.817033 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcbef8b-9fc9-4889-aaf0-1f26a4fa20fd None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:31.863164 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 155/309] 10.222.0.58 () {60 vars in 1134 bytes} [Thu Aug 25 03:00:31 2022] POST /identity/v3/groups => generated 229 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:00:31.886728 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:31.941913 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcbef8b-9fc9-4889-aaf0-1f26a4fa20fd None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:31.982729 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6289b435-f327-42f6-93e1-07e623af1aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:31.983747 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6289b435-f327-42f6-93e1-07e623af1aea None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:31.984209 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6289b435-f327-42f6-93e1-07e623af1aea None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:31.984659 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6289b435-f327-42f6-93e1-07e623af1aea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:31.985607 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6289b435-f327-42f6-93e1-07e623af1aea None admin] RBAC: Adding query filter params (role.id=3de9d2fd0b464c399f67fb5240905596, user.id=d0ab073dde7942f6a951dce04d3822d2, scope.system=all) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:31.986099 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6289b435-f327-42f6-93e1-07e623af1aea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:31.987954 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6289b435-f327-42f6-93e1-07e623af1aea None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:31.994079 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 155/310] 10.222.0.58 () {56 vars in 1308 bytes} [Thu Aug 25 03:00:31 2022] GET /identity/v3/role_assignments?user.id=d0ab073dde7942f6a951dce04d3822d2&scope.system=all&role.id=3de9d2fd0b464c399f67fb5240905596 => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:00:32.218762 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25196401-fd0e-44ba-82f5-ff8334fa3339 None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:32.219478 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25196401-fd0e-44ba-82f5-ff8334fa3339 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:32.219964 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25196401-fd0e-44ba-82f5-ff8334fa3339 None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:32.221507 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 156/311] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:00:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:00:32.227009 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb2760c-cf56-4131-bab4-f91898576c7f None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:32.227296 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb2760c-cf56-4131-bab4-f91898576c7f None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:32.227632 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb2760c-cf56-4131-bab4-f91898576c7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:32.246159 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-edb2760c-cf56-4131-bab4-f91898576c7f None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:00:32.247384 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-edb2760c-cf56-4131-bab4-f91898576c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:32.315623 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 156/312] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:00:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 25 03:00:32.593464 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25196401-fd0e-44ba-82f5-ff8334fa3339 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:32.707049 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25196401-fd0e-44ba-82f5-ff8334fa3339 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:32.742584 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c215b4e9-59ef-4aee-8373-0919fa1dc9e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:32.743464 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c215b4e9-59ef-4aee-8373-0919fa1dc9e2 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:32.743947 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c215b4e9-59ef-4aee-8373-0919fa1dc9e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:32.744462 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c215b4e9-59ef-4aee-8373-0919fa1dc9e2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:32.755033 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c215b4e9-59ef-4aee-8373-0919fa1dc9e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:00:32.760026 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-c215b4e9-59ef-4aee-8373-0919fa1dc9e2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 25 03:00:32.760026 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:00:32.760026 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:00:32.760026 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:00:32.760026 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:00:32.760026 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:00:32.760026 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:00:32.760026 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:00:32.760026 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:00:32.760026 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:00:32.760026 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:00:32.760026 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:00:32.760026 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:00:32.760026 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:00:32.760026 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:00:32.760026 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:00:32.760900 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:00:32.760900 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:00:32.760900 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:00:32.760900 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:00:32.760900 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:00:32.760900 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:00:32.760900 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:00:32.760900 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:00:32.760900 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:00:32.760900 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:00:32.760900 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:00:32.760900 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:00:32.760900 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:00:32.760900 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:00:32.760900 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:00:32.760900 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:00:32.761699 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:00:32.761699 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:00:32.761699 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:00:32.761699 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:00:32.761699 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:00:32.761699 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:00:32.761699 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:00:32.761699 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:00:32.761699 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:00:32.761699 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:00:32.761699 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 25 03:00:32.761699 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:00:32.764220 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 157/313] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Aug 25 03:00:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:00:32.771109 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb2760c-cf56-4131-bab4-f91898576c7f None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:32.824385 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:32.877584 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb2760c-cf56-4131-bab4-f91898576c7f None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:32.913713 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595bfe63-b437-44aa-854f-409bf3447e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:32.915126 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595bfe63-b437-44aa-854f-409bf3447e04 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:32.915633 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595bfe63-b437-44aa-854f-409bf3447e04 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:32.916127 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595bfe63-b437-44aa-854f-409bf3447e04 None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:32.916996 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595bfe63-b437-44aa-854f-409bf3447e04 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:32.917498 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595bfe63-b437-44aa-854f-409bf3447e04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:32.919110 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595bfe63-b437-44aa-854f-409bf3447e04 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:32.928292 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 157/314] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Aug 25 03:00:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:32.935393 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c215b4e9-59ef-4aee-8373-0919fa1dc9e2 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:32.989721 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:33.106230 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c215b4e9-59ef-4aee-8373-0919fa1dc9e2 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:33.143836 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04356fc0-8c4c-4193-b749-7607b07251a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:33.144700 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04356fc0-8c4c-4193-b749-7607b07251a3 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:33.145145 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04356fc0-8c4c-4193-b749-7607b07251a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:33.145599 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04356fc0-8c4c-4193-b749-7607b07251a3 None admin] PATH_INFO: `/v3/users/a644267127e749a9834262937ba37ec4` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:33.146972 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04356fc0-8c4c-4193-b749-7607b07251a3 None admin] RBAC: Authorizing `identity:get_user(user_id=a644267127e749a9834262937ba37ec4)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:33.148664 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04356fc0-8c4c-4193-b749-7607b07251a3 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:33.150856 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 158/315] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Aug 25 03:00:32 2022] GET /identity/v3/users/a644267127e749a9834262937ba37ec4 => generated 297 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:33.162326 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595bfe63-b437-44aa-854f-409bf3447e04 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:33.262568 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595bfe63-b437-44aa-854f-409bf3447e04 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:33.273564 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:33.296821 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db57aae-1df9-4703-83d1-27b14d9b6e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:33.297454 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db57aae-1df9-4703-83d1-27b14d9b6e02 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:33.297740 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db57aae-1df9-4703-83d1-27b14d9b6e02 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:33.298018 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db57aae-1df9-4703-83d1-27b14d9b6e02 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:33.369095 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db57aae-1df9-4703-83d1-27b14d9b6e02 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:33.370580 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db57aae-1df9-4703-83d1-27b14d9b6e02 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:33.402787 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a85636-fcfa-4c6c-be3b-a709827a7336 None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:33.402787 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a85636-fcfa-4c6c-be3b-a709827a7336 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:33.403062 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a85636-fcfa-4c6c-be3b-a709827a7336 None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:33.404583 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 159/316] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:00:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:00:33.409510 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f491595-b5b0-4404-9393-47e9d5e40f85 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:33.409801 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f491595-b5b0-4404-9393-47e9d5e40f85 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:33.410079 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f491595-b5b0-4404-9393-47e9d5e40f85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:33.428694 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-0f491595-b5b0-4404-9393-47e9d5e40f85 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:00:33.429622 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-0f491595-b5b0-4404-9393-47e9d5e40f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:33.497530 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 158/317] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Aug 25 03:00:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 25 03:00:33.499493 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db57aae-1df9-4703-83d1-27b14d9b6e02 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:33.512455 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:33.527934 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 160/318] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:00:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 03:00:33.611119 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db57aae-1df9-4703-83d1-27b14d9b6e02 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:33.647002 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3e7b38-3528-4567-90e4-cccff32183cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:33.647636 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3e7b38-3528-4567-90e4-cccff32183cf None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:33.647911 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3e7b38-3528-4567-90e4-cccff32183cf None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:33.648180 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3e7b38-3528-4567-90e4-cccff32183cf None admin] PATH_INFO: `/v3/projects/54952dee2ab9420d9541e99a1825f5af` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:33.649293 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3e7b38-3528-4567-90e4-cccff32183cf None admin] RBAC: Authorizing `identity:get_project(project_id=54952dee2ab9420d9541e99a1825f5af)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:33.650758 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3e7b38-3528-4567-90e4-cccff32183cf None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:33.652730 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 159/319] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Aug 25 03:00:33 2022] GET /identity/v3/projects/54952dee2ab9420d9541e99a1825f5af => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:00:33.658792 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f491595-b5b0-4404-9393-47e9d5e40f85 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:33.710207 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:33.766618 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f491595-b5b0-4404-9393-47e9d5e40f85 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:33.802869 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b52b0cb-b9e7-4100-9fae-e1a0a9458f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:33.803915 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b52b0cb-b9e7-4100-9fae-e1a0a9458f0a None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:33.804425 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b52b0cb-b9e7-4100-9fae-e1a0a9458f0a None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:33.804893 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b52b0cb-b9e7-4100-9fae-e1a0a9458f0a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:33.805903 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b52b0cb-b9e7-4100-9fae-e1a0a9458f0a None admin] RBAC: Adding query filter params (role.id=3de9d2fd0b464c399f67fb5240905596, scope.project.id=54952dee2ab9420d9541e99a1825f5af, user.id=a644267127e749a9834262937ba37ec4) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:33.806476 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b52b0cb-b9e7-4100-9fae-e1a0a9458f0a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:33.807753 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3e7b38-3528-4567-90e4-cccff32183cf None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:33.808442 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b52b0cb-b9e7-4100-9fae-e1a0a9458f0a None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:33.816221 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 161/320] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Aug 25 03:00:33 2022] GET /identity/v3/role_assignments?user.id=a644267127e749a9834262937ba37ec4&scope.project.id=54952dee2ab9420d9541e99a1825f5af&role.id=3de9d2fd0b464c399f67fb5240905596 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:33.903632 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:33.928073 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3e7b38-3528-4567-90e4-cccff32183cf None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:33.949838 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bcc398-3e2d-48c4-b2c8-5e37fc77b2ec None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:33.950389 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bcc398-3e2d-48c4-b2c8-5e37fc77b2ec None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:33.950492 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bcc398-3e2d-48c4-b2c8-5e37fc77b2ec None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:33.952278 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 162/321] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:00:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:00:33.959367 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffd0969-cdec-49f1-a8ea-ea758e36861f None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:33.960165 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffd0969-cdec-49f1-a8ea-ea758e36861f None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:33.961064 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffd0969-cdec-49f1-a8ea-ea758e36861f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:33.965359 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bd58ea-3e01-49cd-afe5-7fb146ffdb4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:33.966002 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bd58ea-3e01-49cd-afe5-7fb146ffdb4d None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:33.966315 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bd58ea-3e01-49cd-afe5-7fb146ffdb4d None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:33.966577 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bd58ea-3e01-49cd-afe5-7fb146ffdb4d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:33.977836 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bd58ea-3e01-49cd-afe5-7fb146ffdb4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:00:33.983788 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-f4bd58ea-3e01-49cd-afe5-7fb146ffdb4d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 25 03:00:33.983788 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:00:33.983788 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:00:33.983788 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:00:33.983788 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:00:33.983788 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:00:33.983788 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:00:33.983788 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:00:33.983788 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:00:33.983788 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:00:33.983788 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:00:33.983788 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:00:33.983788 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:00:33.983788 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:00:33.983788 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:00:33.983788 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:00:33.994900 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:00:33.994900 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:00:33.994900 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:00:33.994900 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:00:33.994900 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:00:33.994900 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:00:33.994900 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:00:33.994900 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:00:33.994900 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:00:33.994900 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:00:33.994900 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:00:33.994900 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:00:33.994900 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:00:33.994900 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:00:33.994900 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:00:33.994900 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:00:33.995886 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:00:33.995886 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:00:33.995886 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:00:33.995886 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:00:33.995886 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:00:33.995886 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:00:33.995886 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:00:33.995886 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:00:33.995886 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:00:33.995886 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:00:33.995886 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 25 03:00:33.995886 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:00:33.995886 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 160/322] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Aug 25 03:00:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 03:00:33.996540 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-2ffd0969-cdec-49f1-a8ea-ea758e36861f None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:00:33.996540 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-2ffd0969-cdec-49f1-a8ea-ea758e36861f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:33.996671 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bd58ea-3e01-49cd-afe5-7fb146ffdb4d None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:34.071224 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 163/323] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:00:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 03:00:34.072483 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b628327-0600-4520-8421-6913999b7050 None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:34.072765 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b628327-0600-4520-8421-6913999b7050 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:34.073037 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b628327-0600-4520-8421-6913999b7050 None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:34.074261 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 164/324] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:00:34 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:00:34.079838 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da7fa05-d5de-402d-b660-0cc3579204a9 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:34.080125 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da7fa05-d5de-402d-b660-0cc3579204a9 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:34.080396 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da7fa05-d5de-402d-b660-0cc3579204a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:34.101679 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-8da7fa05-d5de-402d-b660-0cc3579204a9 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:00:34.102882 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-8da7fa05-d5de-402d-b660-0cc3579204a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:34.115457 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bd58ea-3e01-49cd-afe5-7fb146ffdb4d None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:34.140774 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:34.151145 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5bd09d-bbb6-4405-a0ca-ecc33dcb71f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:34.151760 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5bd09d-bbb6-4405-a0ca-ecc33dcb71f3 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:34.152043 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5bd09d-bbb6-4405-a0ca-ecc33dcb71f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:34.152329 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5bd09d-bbb6-4405-a0ca-ecc33dcb71f3 None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:34.152994 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5bd09d-bbb6-4405-a0ca-ecc33dcb71f3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:34.153304 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5bd09d-bbb6-4405-a0ca-ecc33dcb71f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:34.154718 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5bd09d-bbb6-4405-a0ca-ecc33dcb71f3 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:34.159857 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:34.164530 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 161/325] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Aug 25 03:00:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:00:34.169954 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5bd09d-bbb6-4405-a0ca-ecc33dcb71f3 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:34.177417 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 165/326] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:00:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 03:00:34.266966 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5bd09d-bbb6-4405-a0ca-ecc33dcb71f3 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:34.299320 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2c2b6a-c432-4c5d-b28b-c3914f33b803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:34.299948 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2c2b6a-c432-4c5d-b28b-c3914f33b803 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:34.300232 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2c2b6a-c432-4c5d-b28b-c3914f33b803 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:34.300508 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2c2b6a-c432-4c5d-b28b-c3914f33b803 None admin] PATH_INFO: `/v3/users/01aa00c8ce8c45fc9946621875d1d064` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:34.301621 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2c2b6a-c432-4c5d-b28b-c3914f33b803 None admin] RBAC: Authorizing `identity:get_user(user_id=01aa00c8ce8c45fc9946621875d1d064)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:34.303091 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2c2b6a-c432-4c5d-b28b-c3914f33b803 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:34.304807 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 162/327] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Aug 25 03:00:34 2022] GET /identity/v3/users/01aa00c8ce8c45fc9946621875d1d064 => generated 293 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:34.310683 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da7fa05-d5de-402d-b660-0cc3579204a9 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:34.343589 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2c2b6a-c432-4c5d-b28b-c3914f33b803 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:34.412127 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da7fa05-d5de-402d-b660-0cc3579204a9 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:34.414881 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:34.448150 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b33775-a6d9-42db-a8d5-67607d40c7d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:34.448755 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b33775-a6d9-42db-a8d5-67607d40c7d0 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:34.449038 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b33775-a6d9-42db-a8d5-67607d40c7d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:34.449313 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b33775-a6d9-42db-a8d5-67607d40c7d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:34.450039 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b33775-a6d9-42db-a8d5-67607d40c7d0 None admin] RBAC: Adding query filter params (role.id=3255f77192524a16bc405173106871fc, user.id=01aa00c8ce8c45fc9946621875d1d064, scope.system=all) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:34.450376 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b33775-a6d9-42db-a8d5-67607d40c7d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:34.450659 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2c2b6a-c432-4c5d-b28b-c3914f33b803 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:34.451867 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b33775-a6d9-42db-a8d5-67607d40c7d0 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:34.463705 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 166/328] 10.222.0.58 () {56 vars in 1308 bytes} [Thu Aug 25 03:00:34 2022] GET /identity/v3/role_assignments?user.id=01aa00c8ce8c45fc9946621875d1d064&scope.system=all&role.id=3255f77192524a16bc405173106871fc => generated 510 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:34.464500 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b33775-a6d9-42db-a8d5-67607d40c7d0 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:34.488756 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569a0f53-ae77-4f31-941b-8c47ef1a5165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:34.489428 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569a0f53-ae77-4f31-941b-8c47ef1a5165 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:34.489721 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569a0f53-ae77-4f31-941b-8c47ef1a5165 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:34.489998 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569a0f53-ae77-4f31-941b-8c47ef1a5165 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:34.502096 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569a0f53-ae77-4f31-941b-8c47ef1a5165 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:00:34.507357 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:34.507534 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-569a0f53-ae77-4f31-941b-8c47ef1a5165 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 25 03:00:34.507534 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:00:34.507534 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:00:34.507534 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:00:34.507534 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:00:34.507534 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:00:34.507534 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:00:34.507534 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:00:34.507534 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:00:34.507534 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:00:34.507534 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:00:34.507534 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:00:34.507534 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:00:34.507534 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:00:34.507534 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:00:34.507534 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:00:34.508345 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:00:34.508345 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:00:34.508345 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:00:34.508345 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:00:34.508345 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:00:34.508345 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:00:34.508345 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:00:34.508345 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:00:34.508345 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:00:34.508345 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:00:34.508345 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:00:34.508345 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:00:34.508345 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:00:34.508345 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:00:34.508345 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:00:34.508345 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:00:34.509113 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:00:34.509113 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:00:34.509113 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:00:34.509113 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:00:34.509113 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:00:34.509113 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:00:34.509113 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:00:34.509113 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:00:34.509113 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:00:34.509113 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:00:34.509113 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 25 03:00:34.509113 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:00:34.513678 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 163/329] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Aug 25 03:00:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 03:00:34.514694 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569a0f53-ae77-4f31-941b-8c47ef1a5165 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:34.592266 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b33775-a6d9-42db-a8d5-67607d40c7d0 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:34.627977 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569a0f53-ae77-4f31-941b-8c47ef1a5165 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:34.637971 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91a850a-d41a-4bc9-a17a-f1fef3e2de53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:34.638874 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91a850a-d41a-4bc9-a17a-f1fef3e2de53 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:34.640915 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91a850a-d41a-4bc9-a17a-f1fef3e2de53 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:34.640915 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91a850a-d41a-4bc9-a17a-f1fef3e2de53 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:34.641737 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91a850a-d41a-4bc9-a17a-f1fef3e2de53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:34.644190 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91a850a-d41a-4bc9-a17a-f1fef3e2de53 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:34.645698 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 167/330] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:00:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:34.651175 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91a850a-d41a-4bc9-a17a-f1fef3e2de53 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:34.660054 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:34.672463 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a961d925-0228-45e0-98d7-4f72cdb52ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:34.673078 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a961d925-0228-45e0-98d7-4f72cdb52ba5 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:34.673360 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a961d925-0228-45e0-98d7-4f72cdb52ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:34.673633 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a961d925-0228-45e0-98d7-4f72cdb52ba5 None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:34.674383 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a961d925-0228-45e0-98d7-4f72cdb52ba5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:34.674639 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a961d925-0228-45e0-98d7-4f72cdb52ba5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:34.676055 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a961d925-0228-45e0-98d7-4f72cdb52ba5 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:34.684113 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 164/331] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Aug 25 03:00:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:34.690433 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a961d925-0228-45e0-98d7-4f72cdb52ba5 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:34.750972 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:34.755170 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91a850a-d41a-4bc9-a17a-f1fef3e2de53 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:34.788214 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a961d925-0228-45e0-98d7-4f72cdb52ba5 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:34.788606 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f69310a-01b5-48c8-b625-ac67f9962967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:34.789277 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f69310a-01b5-48c8-b625-ac67f9962967 None admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:34.789623 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f69310a-01b5-48c8-b625-ac67f9962967 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:34.789926 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f69310a-01b5-48c8-b625-ac67f9962967 None admin] PATH_INFO: `/v3/groups` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:34.790572 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f69310a-01b5-48c8-b625-ac67f9962967 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:34.791995 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f69310a-01b5-48c8-b625-ac67f9962967 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:34.821517 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6530c46d-5abb-4061-8577-ac0b711230ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:34.822146 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6530c46d-5abb-4061-8577-ac0b711230ec None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:34.822454 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6530c46d-5abb-4061-8577-ac0b711230ec None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:34.822724 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6530c46d-5abb-4061-8577-ac0b711230ec None admin] PATH_INFO: `/v3/users/d0ab073dde7942f6a951dce04d3822d2` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:34.823825 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6530c46d-5abb-4061-8577-ac0b711230ec None admin] RBAC: Authorizing `identity:get_user(user_id=d0ab073dde7942f6a951dce04d3822d2)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:34.825242 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6530c46d-5abb-4061-8577-ac0b711230ec None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:34.829032 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 165/332] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Aug 25 03:00:34 2022] GET /identity/v3/users/d0ab073dde7942f6a951dce04d3822d2 => generated 293 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:00:34.832556 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6530c46d-5abb-4061-8577-ac0b711230ec None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:34.885582 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 168/333] 10.222.0.58 () {60 vars in 1134 bytes} [Thu Aug 25 03:00:34 2022] POST /identity/v3/groups => generated 226 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:00:34.892852 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:34.937434 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6530c46d-5abb-4061-8577-ac0b711230ec None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:34.973551 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e48c4f-0a30-4792-94c4-c6466a82c85c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:34.974219 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e48c4f-0a30-4792-94c4-c6466a82c85c None admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:34.974519 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e48c4f-0a30-4792-94c4-c6466a82c85c None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:34.974793 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e48c4f-0a30-4792-94c4-c6466a82c85c None admin] PATH_INFO: `/v3/system/users/d0ab073dde7942f6a951dce04d3822d2/roles/3de9d2fd0b464c399f67fb5240905596` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:34.976237 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e48c4f-0a30-4792-94c4-c6466a82c85c None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=d0ab073dde7942f6a951dce04d3822d2, role_id=3de9d2fd0b464c399f67fb5240905596)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:34.977619 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e48c4f-0a30-4792-94c4-c6466a82c85c None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:35.068867 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 166/334] 10.222.0.58 () {58 vars in 1334 bytes} [Thu Aug 25 03:00:34 2022] PUT /identity/v3/system/users/d0ab073dde7942f6a951dce04d3822d2/roles/3de9d2fd0b464c399f67fb5240905596 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:00:36.996725 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aa9648-7218-4cef-934a-332070a1e337 None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:36.997272 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aa9648-7218-4cef-934a-332070a1e337 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:36.997272 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aa9648-7218-4cef-934a-332070a1e337 None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:36.999957 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 169/335] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:00:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:00:37.005881 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f506873a-eb7b-4757-afa8-197ea00bd5a1 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:37.006399 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f506873a-eb7b-4757-afa8-197ea00bd5a1 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:37.006488 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f506873a-eb7b-4757-afa8-197ea00bd5a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:37.029033 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-f506873a-eb7b-4757-afa8-197ea00bd5a1 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:00:37.030348 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-f506873a-eb7b-4757-afa8-197ea00bd5a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:37.137308 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 167/336] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:00:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 25 03:00:37.230154 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85311ddf-c35f-4e77-95db-f66ddf027e6d None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:37.230611 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85311ddf-c35f-4e77-95db-f66ddf027e6d None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:37.230797 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85311ddf-c35f-4e77-95db-f66ddf027e6d None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:37.232084 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 170/337] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:00:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:00:37.237771 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f529f52c-fad0-442a-b0da-29f4d320568e None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:37.238132 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f529f52c-fad0-442a-b0da-29f4d320568e None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:37.238539 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f529f52c-fad0-442a-b0da-29f4d320568e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:37.265595 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-f529f52c-fad0-442a-b0da-29f4d320568e None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:00:37.270793 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-f529f52c-fad0-442a-b0da-29f4d320568e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:37.367465 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:37.400508 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 168/338] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:00:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 03:00:37.612698 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85311ddf-c35f-4e77-95db-f66ddf027e6d None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:37.800793 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85311ddf-c35f-4e77-95db-f66ddf027e6d None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:37.806134 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a482c34-b442-447a-ab09-cb8dced0e599 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:37.806134 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a482c34-b442-447a-ab09-cb8dced0e599 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:37.806134 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a482c34-b442-447a-ab09-cb8dced0e599 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:37.806762 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 169/339] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:00:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:00:37.838163 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa1e9fe-5b67-4aa4-9dcb-7ec73fbcbc0e None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:37.838697 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa1e9fe-5b67-4aa4-9dcb-7ec73fbcbc0e None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:37.839118 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa1e9fe-5b67-4aa4-9dcb-7ec73fbcbc0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:37.849415 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437e6f4b-1412-43e6-b543-b5040a7a4243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:37.850067 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437e6f4b-1412-43e6-b543-b5040a7a4243 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:37.850381 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437e6f4b-1412-43e6-b543-b5040a7a4243 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:37.850656 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437e6f4b-1412-43e6-b543-b5040a7a4243 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:37.866439 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437e6f4b-1412-43e6-b543-b5040a7a4243 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:00:37.868346 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-437e6f4b-1412-43e6-b543-b5040a7a4243 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 25 03:00:37.868346 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:00:37.868346 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:00:37.868346 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:00:37.868346 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:00:37.868346 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:00:37.868346 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:00:37.868346 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:00:37.868346 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:00:37.868346 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:00:37.868346 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:00:37.868346 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:00:37.868346 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:00:37.868346 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:00:37.868346 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:00:37.868346 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:00:37.869245 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:00:37.869245 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:00:37.869245 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:00:37.869245 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:00:37.869245 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:00:37.869245 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:00:37.869245 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:00:37.869245 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:00:37.869245 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:00:37.869245 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:00:37.869245 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:00:37.869245 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:00:37.869245 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:00:37.869245 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:00:37.869245 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:00:37.869245 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:00:37.870016 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:00:37.870016 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:00:37.870016 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:00:37.870016 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:00:37.870016 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:00:37.870016 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:00:37.870016 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:00:37.870016 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:00:37.870016 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:00:37.870016 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:00:37.870016 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 25 03:00:37.870016 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:00:37.870016 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 171/340] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Aug 25 03:00:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:00:37.879305 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-9aa1e9fe-5b67-4aa4-9dcb-7ec73fbcbc0e None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:00:37.880759 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-9aa1e9fe-5b67-4aa4-9dcb-7ec73fbcbc0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:37.885071 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437e6f4b-1412-43e6-b543-b5040a7a4243 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:37.923173 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:38.021913 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 170/341] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:00:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 03:00:38.022603 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa1e9fe-5b67-4aa4-9dcb-7ec73fbcbc0e None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:38.085020 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437e6f4b-1412-43e6-b543-b5040a7a4243 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:38.155701 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c486fe4f-1ef4-4245-a141-0170ac0ce09d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:38.156313 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c486fe4f-1ef4-4245-a141-0170ac0ce09d None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:38.156591 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c486fe4f-1ef4-4245-a141-0170ac0ce09d None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:38.156857 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c486fe4f-1ef4-4245-a141-0170ac0ce09d None admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:38.157520 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c486fe4f-1ef4-4245-a141-0170ac0ce09d None admin] RBAC: Adding query filter params (name=member) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:38.157817 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c486fe4f-1ef4-4245-a141-0170ac0ce09d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:38.159222 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c486fe4f-1ef4-4245-a141-0170ac0ce09d None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:38.162111 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa1e9fe-5b67-4aa4-9dcb-7ec73fbcbc0e None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:38.180191 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 172/342] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Aug 25 03:00:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:38.181451 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b7de26-0555-465a-b8ae-0ae361171dfa None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:38.181732 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b7de26-0555-465a-b8ae-0ae361171dfa None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:38.182002 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b7de26-0555-465a-b8ae-0ae361171dfa None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:38.183245 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 173/343] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:00:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:00:38.188489 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b7de26-0555-465a-b8ae-0ae361171dfa None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:38.200408 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:38.225492 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918cbeab-de5d-4cd3-a726-7eebce046747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:38.226136 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918cbeab-de5d-4cd3-a726-7eebce046747 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:38.226439 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918cbeab-de5d-4cd3-a726-7eebce046747 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:38.226708 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918cbeab-de5d-4cd3-a726-7eebce046747 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:38.251425 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918cbeab-de5d-4cd3-a726-7eebce046747 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:00:38.256325 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-918cbeab-de5d-4cd3-a726-7eebce046747 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 25 03:00:38.256325 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:00:38.256325 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:00:38.256325 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:00:38.256325 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:00:38.256325 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:00:38.256325 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:00:38.256325 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:00:38.256325 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:00:38.256325 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:00:38.256325 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:00:38.256325 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:00:38.256325 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:00:38.256325 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:00:38.256325 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:00:38.256325 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:00:38.257208 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:00:38.257208 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:00:38.257208 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:00:38.257208 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:00:38.257208 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:00:38.257208 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:00:38.257208 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:00:38.257208 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:00:38.257208 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:00:38.257208 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:00:38.257208 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:00:38.257208 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:00:38.257208 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:00:38.257208 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:00:38.257208 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:00:38.257208 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:00:38.257967 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:00:38.257967 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:00:38.257967 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:00:38.257967 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:00:38.257967 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:00:38.257967 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:00:38.257967 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:00:38.257967 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:00:38.257967 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:00:38.257967 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:00:38.257967 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 25 03:00:38.257967 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:00:38.257967 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 171/344] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Aug 25 03:00:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:00:38.261818 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deaf090-92ec-4cf2-b8e4-c5e139b20a4b None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:38.261818 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deaf090-92ec-4cf2-b8e4-c5e139b20a4b None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:38.261818 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deaf090-92ec-4cf2-b8e4-c5e139b20a4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:38.282220 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-7deaf090-92ec-4cf2-b8e4-c5e139b20a4b None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:00:38.286596 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-7deaf090-92ec-4cf2-b8e4-c5e139b20a4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:38.348941 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:38.382693 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 172/345] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:00:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 03:00:38.383958 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deaf090-92ec-4cf2-b8e4-c5e139b20a4b None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:38.403788 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b7de26-0555-465a-b8ae-0ae361171dfa None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:38.446522 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de529086-4a30-480b-be88-a42bc166e7fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:38.447043 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de529086-4a30-480b-be88-a42bc166e7fd None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:38.447318 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de529086-4a30-480b-be88-a42bc166e7fd None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:38.447642 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de529086-4a30-480b-be88-a42bc166e7fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:38.504301 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deaf090-92ec-4cf2-b8e4-c5e139b20a4b None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:38.505967 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:38.524333 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de529086-4a30-480b-be88-a42bc166e7fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:38.525882 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de529086-4a30-480b-be88-a42bc166e7fd None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:38.565744 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736853a5-9afd-4145-9e68-0facaa0f48a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:38.566462 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736853a5-9afd-4145-9e68-0facaa0f48a3 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:38.566745 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736853a5-9afd-4145-9e68-0facaa0f48a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:38.567016 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736853a5-9afd-4145-9e68-0facaa0f48a3 None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:38.567785 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736853a5-9afd-4145-9e68-0facaa0f48a3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:38.568094 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736853a5-9afd-4145-9e68-0facaa0f48a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:38.569659 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736853a5-9afd-4145-9e68-0facaa0f48a3 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:38.585825 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 173/346] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Aug 25 03:00:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:38.586722 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736853a5-9afd-4145-9e68-0facaa0f48a3 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:38.615113 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:38.640535 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 174/347] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Aug 25 03:00:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 25 03:00:38.641395 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de529086-4a30-480b-be88-a42bc166e7fd None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:38.747642 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de529086-4a30-480b-be88-a42bc166e7fd None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:38.768345 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736853a5-9afd-4145-9e68-0facaa0f48a3 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:38.790436 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92020d18-c3a2-4793-95f3-03a7c5320b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:38.791445 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92020d18-c3a2-4793-95f3-03a7c5320b43 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:38.791730 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92020d18-c3a2-4793-95f3-03a7c5320b43 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:38.792003 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92020d18-c3a2-4793-95f3-03a7c5320b43 None admin] PATH_INFO: `/v3/users/d0ab073dde7942f6a951dce04d3822d2` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:38.793283 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92020d18-c3a2-4793-95f3-03a7c5320b43 None admin] RBAC: Authorizing `identity:get_user(user_id=d0ab073dde7942f6a951dce04d3822d2)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:38.794910 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92020d18-c3a2-4793-95f3-03a7c5320b43 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:38.797457 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 175/348] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Aug 25 03:00:38 2022] GET /identity/v3/users/d0ab073dde7942f6a951dce04d3822d2 => generated 293 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:38.798220 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92020d18-c3a2-4793-95f3-03a7c5320b43 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:38.828299 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9824f216-75ee-4ec7-ad94-f13e8f394289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:38.829325 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9824f216-75ee-4ec7-ad94-f13e8f394289 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:38.829806 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9824f216-75ee-4ec7-ad94-f13e8f394289 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:38.830326 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9824f216-75ee-4ec7-ad94-f13e8f394289 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:38.843533 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9824f216-75ee-4ec7-ad94-f13e8f394289 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:00:38.849348 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-9824f216-75ee-4ec7-ad94-f13e8f394289 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 25 03:00:38.849348 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:00:38.849348 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:00:38.849348 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:00:38.849348 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:00:38.849348 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:00:38.849348 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:00:38.849348 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:00:38.849348 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:00:38.849348 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:00:38.849348 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:00:38.849348 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:00:38.849348 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:00:38.849348 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:00:38.849348 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:00:38.849348 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:00:38.852690 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:00:38.852690 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:00:38.852690 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:00:38.852690 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:00:38.852690 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:00:38.852690 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:00:38.852690 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:00:38.852690 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:00:38.852690 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:00:38.852690 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:00:38.852690 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:00:38.852690 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:00:38.852690 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:00:38.852690 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:00:38.852690 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:00:38.852690 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:00:38.853521 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:00:38.853521 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:00:38.853521 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:00:38.853521 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:00:38.853521 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:00:38.853521 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:00:38.853521 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:00:38.853521 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:00:38.853521 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:00:38.853521 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:00:38.853521 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 25 03:00:38.853521 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:00:38.856177 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 174/349] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Aug 25 03:00:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:00:38.857292 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9824f216-75ee-4ec7-ad94-f13e8f394289 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:38.866711 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:38.915596 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:38.940092 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92020d18-c3a2-4793-95f3-03a7c5320b43 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:38.989306 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12317194-6e5f-49a8-9261-c29ef6f96d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:38.989943 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12317194-6e5f-49a8-9261-c29ef6f96d8d None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:38.990242 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12317194-6e5f-49a8-9261-c29ef6f96d8d None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:38.990557 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12317194-6e5f-49a8-9261-c29ef6f96d8d None admin] PATH_INFO: `/v3/projects/0d32389226ec4d2dac8c390a33d5436c` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:38.991751 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12317194-6e5f-49a8-9261-c29ef6f96d8d None admin] RBAC: Authorizing `identity:get_project(project_id=0d32389226ec4d2dac8c390a33d5436c)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:38.993198 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12317194-6e5f-49a8-9261-c29ef6f96d8d None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:38.995542 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 176/350] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Aug 25 03:00:38 2022] GET /identity/v3/projects/0d32389226ec4d2dac8c390a33d5436c => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:38.996141 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12317194-6e5f-49a8-9261-c29ef6f96d8d None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:39.033492 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9824f216-75ee-4ec7-ad94-f13e8f394289 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:39.075133 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19dfb59-9953-4e0e-9487-e1ae594c8efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:39.075759 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19dfb59-9953-4e0e-9487-e1ae594c8efa None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:39.078330 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19dfb59-9953-4e0e-9487-e1ae594c8efa None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:39.078330 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19dfb59-9953-4e0e-9487-e1ae594c8efa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:39.078330 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19dfb59-9953-4e0e-9487-e1ae594c8efa None admin] RBAC: Adding query filter params (role.id=3de9d2fd0b464c399f67fb5240905596, user.id=d0ab073dde7942f6a951dce04d3822d2, scope.system=all) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:39.078574 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19dfb59-9953-4e0e-9487-e1ae594c8efa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:39.080190 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19dfb59-9953-4e0e-9487-e1ae594c8efa None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:39.097315 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 175/351] 10.222.0.58 () {56 vars in 1308 bytes} [Thu Aug 25 03:00:38 2022] GET /identity/v3/role_assignments?user.id=d0ab073dde7942f6a951dce04d3822d2&scope.system=all&role.id=3de9d2fd0b464c399f67fb5240905596 => generated 510 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:39.098261 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19dfb59-9953-4e0e-9487-e1ae594c8efa None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:39.136895 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12317194-6e5f-49a8-9261-c29ef6f96d8d None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:39.157303 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:39.178659 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cae81e7-82e3-4479-ac7d-ebfe8ae7fdf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:39.179967 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cae81e7-82e3-4479-ac7d-ebfe8ae7fdf0 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:39.179967 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cae81e7-82e3-4479-ac7d-ebfe8ae7fdf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:39.179967 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cae81e7-82e3-4479-ac7d-ebfe8ae7fdf0 None admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:39.183608 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cae81e7-82e3-4479-ac7d-ebfe8ae7fdf0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:39.183608 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cae81e7-82e3-4479-ac7d-ebfe8ae7fdf0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:39.183608 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cae81e7-82e3-4479-ac7d-ebfe8ae7fdf0 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:39.193068 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 177/352] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Aug 25 03:00:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:39.193957 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cae81e7-82e3-4479-ac7d-ebfe8ae7fdf0 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:39.213308 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:39.234423 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19dfb59-9953-4e0e-9487-e1ae594c8efa None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:39.275625 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b781fe21-9188-4ca5-b932-dfaa989f19a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:39.276610 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b781fe21-9188-4ca5-b932-dfaa989f19a7 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:39.276889 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b781fe21-9188-4ca5-b932-dfaa989f19a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:39.277160 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b781fe21-9188-4ca5-b932-dfaa989f19a7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:39.290614 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b781fe21-9188-4ca5-b932-dfaa989f19a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:00:39.295689 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-b781fe21-9188-4ca5-b932-dfaa989f19a7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 25 03:00:39.295689 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:00:39.295689 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:00:39.295689 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:00:39.295689 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:00:39.295689 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:00:39.295689 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:00:39.295689 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:00:39.295689 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:00:39.295689 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:00:39.295689 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:00:39.295689 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:00:39.295689 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:00:39.295689 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:00:39.295689 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:00:39.295689 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:00:39.296585 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:00:39.296585 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:00:39.296585 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:00:39.296585 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:00:39.296585 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:00:39.296585 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:00:39.296585 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:00:39.296585 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:00:39.296585 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:00:39.296585 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:00:39.296585 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:00:39.296585 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:00:39.296585 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:00:39.296585 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:00:39.296585 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:00:39.296585 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:00:39.297336 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:00:39.297336 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:00:39.297336 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:00:39.297336 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:00:39.297336 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:00:39.297336 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:00:39.297336 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:00:39.297336 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:00:39.297336 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:00:39.297336 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:00:39.297336 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 25 03:00:39.297336 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:00:39.297336 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 176/353] 10.222.0.58 () {56 vars in 1114 bytes} [Thu Aug 25 03:00:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:00:39.297971 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196d2deb-fcf6-437f-8214-e91db8716e30 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:39.298590 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196d2deb-fcf6-437f-8214-e91db8716e30 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:39.298687 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196d2deb-fcf6-437f-8214-e91db8716e30 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:39.300467 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 177/354] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:00:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:00:39.300927 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196d2deb-fcf6-437f-8214-e91db8716e30 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:39.325524 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cae81e7-82e3-4479-ac7d-ebfe8ae7fdf0 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:39.365035 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e558b45d-0eb1-4631-89d0-ce359a9399fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:39.366158 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e558b45d-0eb1-4631-89d0-ce359a9399fc None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:39.366984 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e558b45d-0eb1-4631-89d0-ce359a9399fc None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:39.367496 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e558b45d-0eb1-4631-89d0-ce359a9399fc None admin] PATH_INFO: `/v3/groups/6e496f4f230c4815b5e45878999d27cf` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:39.376176 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e558b45d-0eb1-4631-89d0-ce359a9399fc None admin] RBAC: Authorizing `identity:get_group(group_id=6e496f4f230c4815b5e45878999d27cf)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:39.378181 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e558b45d-0eb1-4631-89d0-ce359a9399fc None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:39.380439 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 178/355] 10.222.0.58 () {56 vars in 1180 bytes} [Thu Aug 25 03:00:39 2022] GET /identity/v3/groups/6e496f4f230c4815b5e45878999d27cf => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:39.381960 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e558b45d-0eb1-4631-89d0-ce359a9399fc None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:39.438658 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:39.449489 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196d2deb-fcf6-437f-8214-e91db8716e30 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:39.497438 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d269b401-80db-4ac8-8584-24931eef1b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:39.498052 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d269b401-80db-4ac8-8584-24931eef1b4b None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:39.498352 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d269b401-80db-4ac8-8584-24931eef1b4b None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:39.499260 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d269b401-80db-4ac8-8584-24931eef1b4b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:39.512353 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:39.515469 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e558b45d-0eb1-4631-89d0-ce359a9399fc None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:39.559998 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38a6893-cbda-4c50-91c3-6b7778c80477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:39.560536 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38a6893-cbda-4c50-91c3-6b7778c80477 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:39.560822 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38a6893-cbda-4c50-91c3-6b7778c80477 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:39.561722 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38a6893-cbda-4c50-91c3-6b7778c80477 None admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:39.562337 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38a6893-cbda-4c50-91c3-6b7778c80477 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:39.562646 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38a6893-cbda-4c50-91c3-6b7778c80477 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:39.564043 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38a6893-cbda-4c50-91c3-6b7778c80477 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:39.572981 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 179/356] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Aug 25 03:00:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:39.574183 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d533e92-b2ea-4cdf-b4c4-243e96161a97 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:39.575404 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d533e92-b2ea-4cdf-b4c4-243e96161a97 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:39.575687 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d533e92-b2ea-4cdf-b4c4-243e96161a97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:39.583182 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d269b401-80db-4ac8-8584-24931eef1b4b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:39.585811 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d269b401-80db-4ac8-8584-24931eef1b4b None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:39.597673 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-9d533e92-b2ea-4cdf-b4c4-243e96161a97 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:00:39.598926 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-9d533e92-b2ea-4cdf-b4c4-243e96161a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:39.679328 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 180/357] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:00:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 03:00:39.680058 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d533e92-b2ea-4cdf-b4c4-243e96161a97 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:39.701980 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 178/358] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Aug 25 03:00:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 25 03:00:39.703692 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf707f6-23f3-4872-8e6d-fef11c2f3000 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:39.704024 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf707f6-23f3-4872-8e6d-fef11c2f3000 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:39.704298 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf707f6-23f3-4872-8e6d-fef11c2f3000 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:39.705984 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 179/359] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:00:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:00:39.706580 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf707f6-23f3-4872-8e6d-fef11c2f3000 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:39.723597 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:39.766507 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:39.813392 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d533e92-b2ea-4cdf-b4c4-243e96161a97 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:39.841802 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf707f6-23f3-4872-8e6d-fef11c2f3000 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:39.854778 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab48b95-dda4-45a4-9f1d-b32e694e830e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:39.855450 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab48b95-dda4-45a4-9f1d-b32e694e830e None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:39.855730 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab48b95-dda4-45a4-9f1d-b32e694e830e None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:39.856059 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab48b95-dda4-45a4-9f1d-b32e694e830e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:39.856806 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab48b95-dda4-45a4-9f1d-b32e694e830e None admin] RBAC: Adding query filter params (group.id=6e496f4f230c4815b5e45878999d27cf, role.id=3255f77192524a16bc405173106871fc, scope.project.id=0d32389226ec4d2dac8c390a33d5436c) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:39.857115 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab48b95-dda4-45a4-9f1d-b32e694e830e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:39.859121 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab48b95-dda4-45a4-9f1d-b32e694e830e None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:39.865456 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 181/360] 10.222.0.58 () {56 vars in 1376 bytes} [Thu Aug 25 03:00:39 2022] GET /identity/v3/role_assignments?group.id=6e496f4f230c4815b5e45878999d27cf&scope.project.id=0d32389226ec4d2dac8c390a33d5436c&role.id=3255f77192524a16bc405173106871fc => generated 260 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:39.866524 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab48b95-dda4-45a4-9f1d-b32e694e830e None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:39.887251 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622b2246-4e12-416c-bd09-d5f1b2158069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:39.887891 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622b2246-4e12-416c-bd09-d5f1b2158069 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:39.888229 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622b2246-4e12-416c-bd09-d5f1b2158069 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:39.888666 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622b2246-4e12-416c-bd09-d5f1b2158069 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:39.968664 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622b2246-4e12-416c-bd09-d5f1b2158069 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:39.970435 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622b2246-4e12-416c-bd09-d5f1b2158069 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:39.989215 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:39.993338 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab48b95-dda4-45a4-9f1d-b32e694e830e None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:40.027786 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:40.041455 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627c2d91-a4ee-4643-86c4-d48d61260e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:40.041455 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627c2d91-a4ee-4643-86c4-d48d61260e86 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:40.041731 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627c2d91-a4ee-4643-86c4-d48d61260e86 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:40.042023 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627c2d91-a4ee-4643-86c4-d48d61260e86 None admin] PATH_INFO: `/v3/projects/54952dee2ab9420d9541e99a1825f5af` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:40.043342 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627c2d91-a4ee-4643-86c4-d48d61260e86 None admin] RBAC: Authorizing `identity:get_project(project_id=54952dee2ab9420d9541e99a1825f5af)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:40.044938 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627c2d91-a4ee-4643-86c4-d48d61260e86 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:40.047614 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 182/361] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Aug 25 03:00:39 2022] GET /identity/v3/projects/54952dee2ab9420d9541e99a1825f5af => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:40.048865 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903b04dc-8cb9-4fdf-b88b-1933c54426e2 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:40.049157 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903b04dc-8cb9-4fdf-b88b-1933c54426e2 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:40.049435 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903b04dc-8cb9-4fdf-b88b-1933c54426e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:40.074583 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-903b04dc-8cb9-4fdf-b88b-1933c54426e2 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:00:40.075694 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-903b04dc-8cb9-4fdf-b88b-1933c54426e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:40.112878 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 180/362] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Aug 25 03:00:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 25 03:00:40.115413 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622b2246-4e12-416c-bd09-d5f1b2158069 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:40.168169 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 183/363] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:00:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 03:00:40.169024 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903b04dc-8cb9-4fdf-b88b-1933c54426e2 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:40.246548 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622b2246-4e12-416c-bd09-d5f1b2158069 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:40.292099 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:40.294021 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a52919-c071-4cee-8a0a-2691c18c6cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:40.294944 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a52919-c071-4cee-8a0a-2691c18c6cec None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:40.295407 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a52919-c071-4cee-8a0a-2691c18c6cec None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:40.296406 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a52919-c071-4cee-8a0a-2691c18c6cec None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:40.299150 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903b04dc-8cb9-4fdf-b88b-1933c54426e2 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:40.307751 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a52919-c071-4cee-8a0a-2691c18c6cec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:00:40.312888 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-94a52919-c071-4cee-8a0a-2691c18c6cec None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 25 03:00:40.312888 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:00:40.312888 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:00:40.312888 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:00:40.312888 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:00:40.312888 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:00:40.312888 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:00:40.312888 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:00:40.312888 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:00:40.312888 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:00:40.312888 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:00:40.312888 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:00:40.312888 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:00:40.312888 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:00:40.312888 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:00:40.312888 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:00:40.313809 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:00:40.313809 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:00:40.313809 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:00:40.313809 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:00:40.313809 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:00:40.313809 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:00:40.313809 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:00:40.313809 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:00:40.313809 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:00:40.313809 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:00:40.313809 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:00:40.313809 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:00:40.313809 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:00:40.313809 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:00:40.313809 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:00:40.313809 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:00:40.314637 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:00:40.314637 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:00:40.314637 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:00:40.314637 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:00:40.314637 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:00:40.314637 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:00:40.314637 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:00:40.314637 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:00:40.314637 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:00:40.314637 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:00:40.314637 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 25 03:00:40.314637 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:00:40.316961 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 181/364] 10.222.0.58 () {56 vars in 1114 bytes} [Thu Aug 25 03:00:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:00:40.317991 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a52919-c071-4cee-8a0a-2691c18c6cec None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:40.321565 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:40.340642 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bae37f8-8e21-4c0f-a6b0-b3ac00e88a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:40.341357 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bae37f8-8e21-4c0f-a6b0-b3ac00e88a7d None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:40.341746 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bae37f8-8e21-4c0f-a6b0-b3ac00e88a7d None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:40.342118 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bae37f8-8e21-4c0f-a6b0-b3ac00e88a7d None admin] PATH_INFO: `/v3/groups/6e496f4f230c4815b5e45878999d27cf` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:40.343897 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bae37f8-8e21-4c0f-a6b0-b3ac00e88a7d None admin] RBAC: Authorizing `identity:get_group(group_id=6e496f4f230c4815b5e45878999d27cf)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:40.345813 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bae37f8-8e21-4c0f-a6b0-b3ac00e88a7d None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:40.351440 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 184/365] 10.222.0.58 () {56 vars in 1180 bytes} [Thu Aug 25 03:00:40 2022] GET /identity/v3/groups/6e496f4f230c4815b5e45878999d27cf => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:40.351440 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bae37f8-8e21-4c0f-a6b0-b3ac00e88a7d None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:40.451574 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a52919-c071-4cee-8a0a-2691c18c6cec None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:40.464563 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bae37f8-8e21-4c0f-a6b0-b3ac00e88a7d None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:40.490607 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d493072d-4332-42c3-b94d-f430fdccb365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:40.491516 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d493072d-4332-42c3-b94d-f430fdccb365 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:40.491941 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d493072d-4332-42c3-b94d-f430fdccb365 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:40.492362 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d493072d-4332-42c3-b94d-f430fdccb365 None admin] PATH_INFO: `/v3/projects/0d32389226ec4d2dac8c390a33d5436c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:40.493842 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d493072d-4332-42c3-b94d-f430fdccb365 None admin] RBAC: Authorizing `identity:get_project(project_id=0d32389226ec4d2dac8c390a33d5436c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:40.495515 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d493072d-4332-42c3-b94d-f430fdccb365 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:40.497573 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 182/366] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Aug 25 03:00:40 2022] GET /identity/v3/projects/0d32389226ec4d2dac8c390a33d5436c => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:40.498816 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d493072d-4332-42c3-b94d-f430fdccb365 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:40.502479 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf04fe21-f372-4c5d-8499-73a7e525da2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:40.503830 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf04fe21-f372-4c5d-8499-73a7e525da2b None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:40.503830 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf04fe21-f372-4c5d-8499-73a7e525da2b None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:40.503830 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf04fe21-f372-4c5d-8499-73a7e525da2b None admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:40.504505 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf04fe21-f372-4c5d-8499-73a7e525da2b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:40.504912 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf04fe21-f372-4c5d-8499-73a7e525da2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:40.506487 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf04fe21-f372-4c5d-8499-73a7e525da2b None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:40.516816 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 185/367] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Aug 25 03:00:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:40.517342 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf04fe21-f372-4c5d-8499-73a7e525da2b None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:40.551916 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:40.596267 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:40.643248 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d493072d-4332-42c3-b94d-f430fdccb365 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:40.652175 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf04fe21-f372-4c5d-8499-73a7e525da2b None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:40.691336 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251e34fa-edc8-43ce-9949-ae56525f233d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:40.691835 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e12f28-0be5-4bdc-ae25-1a7aadcb35c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:40.692322 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251e34fa-edc8-43ce-9949-ae56525f233d None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:40.692495 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e12f28-0be5-4bdc-ae25-1a7aadcb35c5 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:40.692590 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251e34fa-edc8-43ce-9949-ae56525f233d None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:40.692767 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e12f28-0be5-4bdc-ae25-1a7aadcb35c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:40.692862 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251e34fa-edc8-43ce-9949-ae56525f233d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:40.693014 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e12f28-0be5-4bdc-ae25-1a7aadcb35c5 None admin] PATH_INFO: `/v3/groups/6e496f4f230c4815b5e45878999d27cf` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:40.693584 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251e34fa-edc8-43ce-9949-ae56525f233d None admin] RBAC: Adding query filter params (group.id=6e496f4f230c4815b5e45878999d27cf, role.id=3255f77192524a16bc405173106871fc, scope.project.id=54952dee2ab9420d9541e99a1825f5af) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:40.693894 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251e34fa-edc8-43ce-9949-ae56525f233d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:40.697261 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e12f28-0be5-4bdc-ae25-1a7aadcb35c5 None admin] RBAC: Authorizing `identity:get_group(group_id=6e496f4f230c4815b5e45878999d27cf)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:40.697379 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251e34fa-edc8-43ce-9949-ae56525f233d None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:40.697453 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e12f28-0be5-4bdc-ae25-1a7aadcb35c5 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:40.697988 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 186/368] 10.222.0.58 () {56 vars in 1180 bytes} [Thu Aug 25 03:00:40 2022] GET /identity/v3/groups/6e496f4f230c4815b5e45878999d27cf => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:40.702577 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e12f28-0be5-4bdc-ae25-1a7aadcb35c5 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:40.708549 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 183/369] 10.222.0.58 () {56 vars in 1376 bytes} [Thu Aug 25 03:00:40 2022] GET /identity/v3/role_assignments?group.id=6e496f4f230c4815b5e45878999d27cf&scope.project.id=54952dee2ab9420d9541e99a1825f5af&role.id=3255f77192524a16bc405173106871fc => generated 260 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:40.709403 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251e34fa-edc8-43ce-9949-ae56525f233d None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:40.830918 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:40.838629 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e12f28-0be5-4bdc-ae25-1a7aadcb35c5 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:40.852408 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251e34fa-edc8-43ce-9949-ae56525f233d None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:40.881559 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b2692d-059c-4b80-b089-264520cb7091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:40.882355 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b2692d-059c-4b80-b089-264520cb7091 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:40.882642 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b2692d-059c-4b80-b089-264520cb7091 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:40.882919 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b2692d-059c-4b80-b089-264520cb7091 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:40.897514 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b2692d-059c-4b80-b089-264520cb7091 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:00:40.899226 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247d96e2-2666-46b5-b0e9-6ea0beb68998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:40.899849 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247d96e2-2666-46b5-b0e9-6ea0beb68998 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:40.900130 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247d96e2-2666-46b5-b0e9-6ea0beb68998 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:40.900405 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247d96e2-2666-46b5-b0e9-6ea0beb68998 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:40.902535 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-e6b2692d-059c-4b80-b089-264520cb7091 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 03:00:40.902535 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:00:40.902535 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:00:40.902535 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:00:40.902535 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:00:40.902535 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:00:40.902535 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:00:40.902535 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:00:40.902535 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:00:40.902535 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:00:40.902535 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:00:40.902535 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:00:40.902535 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:00:40.902535 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:00:40.902535 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:00:40.902535 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:00:40.903402 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:00:40.903402 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:00:40.903402 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:00:40.903402 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:00:40.903402 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:00:40.903402 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:00:40.903402 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:00:40.903402 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:00:40.903402 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:00:40.903402 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:00:40.903402 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:00:40.903402 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:00:40.903402 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:00:40.903402 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:00:40.903402 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:00:40.903402 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:00:40.904254 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:00:40.904254 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:00:40.904254 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:00:40.904254 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:00:40.904254 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:00:40.904254 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:00:40.904254 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:00:40.904254 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:00:40.904254 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:00:40.904254 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:00:40.904254 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 03:00:40.904254 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:00:40.904862 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:40.904964 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 187/370] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Aug 25 03:00:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:00:40.905561 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b2692d-059c-4b80-b089-264520cb7091 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:40.990140 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247d96e2-2666-46b5-b0e9-6ea0beb68998 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:40.991653 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247d96e2-2666-46b5-b0e9-6ea0beb68998 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:41.029426 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b2692d-059c-4b80-b089-264520cb7091 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:41.072331 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660d637a-c7a5-4126-b931-1daa3864e383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:41.072700 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660d637a-c7a5-4126-b931-1daa3864e383 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:41.072867 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660d637a-c7a5-4126-b931-1daa3864e383 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:41.073129 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660d637a-c7a5-4126-b931-1daa3864e383 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:41.073870 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660d637a-c7a5-4126-b931-1daa3864e383 None admin] RBAC: Adding query filter params (group.id=6e496f4f230c4815b5e45878999d27cf, role.id=7886cc7e333a42d08085932494a3ea6b, scope.project.id=0d32389226ec4d2dac8c390a33d5436c) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:41.074180 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660d637a-c7a5-4126-b931-1daa3864e383 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:41.076333 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660d637a-c7a5-4126-b931-1daa3864e383 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:41.086651 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 188/371] 10.222.0.58 () {56 vars in 1376 bytes} [Thu Aug 25 03:00:40 2022] GET /identity/v3/role_assignments?group.id=6e496f4f230c4815b5e45878999d27cf&scope.project.id=0d32389226ec4d2dac8c390a33d5436c&role.id=7886cc7e333a42d08085932494a3ea6b => generated 260 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:41.087581 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660d637a-c7a5-4126-b931-1daa3864e383 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:41.112753 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:41.140193 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 184/372] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Aug 25 03:00:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 25 03:00:41.148493 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247d96e2-2666-46b5-b0e9-6ea0beb68998 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:41.193613 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:41.265338 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660d637a-c7a5-4126-b931-1daa3864e383 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:41.287092 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247d96e2-2666-46b5-b0e9-6ea0beb68998 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:41.322294 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17ac31b-b713-4a02-9019-a0ece018dae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:41.323255 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ac31b-b713-4a02-9019-a0ece018dae2 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:41.323730 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ac31b-b713-4a02-9019-a0ece018dae2 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:41.324233 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ac31b-b713-4a02-9019-a0ece018dae2 None admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:41.325113 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17ac31b-b713-4a02-9019-a0ece018dae2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:41.326640 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17ac31b-b713-4a02-9019-a0ece018dae2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:41.329030 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17ac31b-b713-4a02-9019-a0ece018dae2 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:41.343264 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 189/373] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Aug 25 03:00:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:41.349185 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17ac31b-b713-4a02-9019-a0ece018dae2 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:41.352440 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8a5d63-ff53-4148-892e-f37c04fad7bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:41.353086 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8a5d63-ff53-4148-892e-f37c04fad7bf None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:41.354003 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8a5d63-ff53-4148-892e-f37c04fad7bf None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:41.354003 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8a5d63-ff53-4148-892e-f37c04fad7bf None admin] PATH_INFO: `/v3/projects/54952dee2ab9420d9541e99a1825f5af` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:41.356726 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8a5d63-ff53-4148-892e-f37c04fad7bf None admin] RBAC: Authorizing `identity:get_project(project_id=54952dee2ab9420d9541e99a1825f5af)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:41.358217 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8a5d63-ff53-4148-892e-f37c04fad7bf None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:41.362359 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 185/374] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Aug 25 03:00:41 2022] GET /identity/v3/projects/54952dee2ab9420d9541e99a1825f5af => generated 302 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:00:41.368391 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8a5d63-ff53-4148-892e-f37c04fad7bf None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:41.457161 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:41.479240 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17ac31b-b713-4a02-9019-a0ece018dae2 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:41.521381 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd36d85-e257-4b70-a74b-a79835fbdc1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:41.522029 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd36d85-e257-4b70-a74b-a79835fbdc1d None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:41.529410 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd36d85-e257-4b70-a74b-a79835fbdc1d None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:41.529707 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd36d85-e257-4b70-a74b-a79835fbdc1d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:41.543040 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8a5d63-ff53-4148-892e-f37c04fad7bf None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:41.552987 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:41.590620 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6293e1fb-097b-477f-b9ec-841f164d7eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:41.591655 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6293e1fb-097b-477f-b9ec-841f164d7eee None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:41.592129 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6293e1fb-097b-477f-b9ec-841f164d7eee None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:41.592618 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6293e1fb-097b-477f-b9ec-841f164d7eee None admin] PATH_INFO: `/v3/groups/6e496f4f230c4815b5e45878999d27cf` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:41.593944 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6293e1fb-097b-477f-b9ec-841f164d7eee None admin] RBAC: Authorizing `identity:get_group(group_id=6e496f4f230c4815b5e45878999d27cf)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:41.595634 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6293e1fb-097b-477f-b9ec-841f164d7eee None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:41.597697 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 186/375] 10.222.0.58 () {56 vars in 1180 bytes} [Thu Aug 25 03:00:41 2022] GET /identity/v3/groups/6e496f4f230c4815b5e45878999d27cf => generated 226 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:41.603367 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6293e1fb-097b-477f-b9ec-841f164d7eee None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:41.605066 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd36d85-e257-4b70-a74b-a79835fbdc1d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:41.606654 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd36d85-e257-4b70-a74b-a79835fbdc1d None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:41.693491 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:41.716211 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 190/376] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Aug 25 03:00:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 25 03:00:41.716820 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6293e1fb-097b-477f-b9ec-841f164d7eee None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:41.721994 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd36d85-e257-4b70-a74b-a79835fbdc1d None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:41.754471 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f5ce36-d231-4459-a7f3-fc2364d0a9d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:41.755110 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f5ce36-d231-4459-a7f3-fc2364d0a9d5 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:41.755389 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f5ce36-d231-4459-a7f3-fc2364d0a9d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:41.755659 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f5ce36-d231-4459-a7f3-fc2364d0a9d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:41.756384 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f5ce36-d231-4459-a7f3-fc2364d0a9d5 None admin] RBAC: Adding query filter params (group.id=6e496f4f230c4815b5e45878999d27cf, role.id=7886cc7e333a42d08085932494a3ea6b, scope.project.id=54952dee2ab9420d9541e99a1825f5af) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:41.756687 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f5ce36-d231-4459-a7f3-fc2364d0a9d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:41.758123 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f5ce36-d231-4459-a7f3-fc2364d0a9d5 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:41.764183 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 187/377] 10.222.0.58 () {56 vars in 1376 bytes} [Thu Aug 25 03:00:41 2022] GET /identity/v3/role_assignments?group.id=6e496f4f230c4815b5e45878999d27cf&scope.project.id=54952dee2ab9420d9541e99a1825f5af&role.id=7886cc7e333a42d08085932494a3ea6b => generated 260 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:41.862261 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd36d85-e257-4b70-a74b-a79835fbdc1d None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:41.901253 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773017b2-ff9a-44dd-b876-350a0a4ec64a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:41.901807 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773017b2-ff9a-44dd-b876-350a0a4ec64a None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:41.902085 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773017b2-ff9a-44dd-b876-350a0a4ec64a None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:41.902386 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773017b2-ff9a-44dd-b876-350a0a4ec64a None admin] PATH_INFO: `/v3/projects/b981090d110a4dd99327dbd654b9daa9` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:41.903747 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773017b2-ff9a-44dd-b876-350a0a4ec64a None admin] RBAC: Authorizing `identity:get_project(project_id=b981090d110a4dd99327dbd654b9daa9)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:41.905327 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773017b2-ff9a-44dd-b876-350a0a4ec64a None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:41.907354 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 191/378] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Aug 25 03:00:41 2022] GET /identity/v3/projects/b981090d110a4dd99327dbd654b9daa9 => generated 344 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:00:41.912672 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f5ce36-d231-4459-a7f3-fc2364d0a9d5 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:41.958811 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:42.034868 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f5ce36-d231-4459-a7f3-fc2364d0a9d5 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:42.074869 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e001bb-e0ae-4a38-baa0-ce4dad0bbfd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:42.075516 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e001bb-e0ae-4a38-baa0-ce4dad0bbfd8 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:42.075799 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e001bb-e0ae-4a38-baa0-ce4dad0bbfd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:42.076075 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e001bb-e0ae-4a38-baa0-ce4dad0bbfd8 None admin] PATH_INFO: `/v3/groups/f1ea6bd0777d45e896a30daf9eaed344` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:42.082670 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e001bb-e0ae-4a38-baa0-ce4dad0bbfd8 None admin] RBAC: Authorizing `identity:get_group(group_id=f1ea6bd0777d45e896a30daf9eaed344)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:42.084119 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e001bb-e0ae-4a38-baa0-ce4dad0bbfd8 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:42.088322 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 188/379] 10.222.0.58 () {56 vars in 1180 bytes} [Thu Aug 25 03:00:41 2022] GET /identity/v3/groups/f1ea6bd0777d45e896a30daf9eaed344 => generated 229 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:00:42.091660 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773017b2-ff9a-44dd-b876-350a0a4ec64a None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:42.142969 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:42.203200 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773017b2-ff9a-44dd-b876-350a0a4ec64a None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:42.212043 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d253537-73cc-4a12-a639-4482e0f6a0d9 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:42.212560 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d253537-73cc-4a12-a639-4482e0f6a0d9 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:42.212678 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d253537-73cc-4a12-a639-4482e0f6a0d9 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:42.214385 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 189/380] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:00:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:00:42.221377 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f59f053-9140-4676-b714-7dfbaadcba9c None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:42.221689 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f59f053-9140-4676-b714-7dfbaadcba9c None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:42.221918 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f59f053-9140-4676-b714-7dfbaadcba9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:42.248516 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-1f59f053-9140-4676-b714-7dfbaadcba9c None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:00:42.250317 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-1f59f053-9140-4676-b714-7dfbaadcba9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:42.272574 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09471607-a177-4fc5-99a5-26b7aad4878a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:42.273205 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09471607-a177-4fc5-99a5-26b7aad4878a None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:42.273463 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09471607-a177-4fc5-99a5-26b7aad4878a None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:42.273742 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09471607-a177-4fc5-99a5-26b7aad4878a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:42.274501 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09471607-a177-4fc5-99a5-26b7aad4878a None admin] RBAC: Adding query filter params (group.id=f1ea6bd0777d45e896a30daf9eaed344, role.id=62f19ea272b447ccaab1576cd9bf4fd9, scope.project.id=b981090d110a4dd99327dbd654b9daa9) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:42.274812 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09471607-a177-4fc5-99a5-26b7aad4878a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:42.276269 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09471607-a177-4fc5-99a5-26b7aad4878a None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:42.291413 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 192/381] 10.222.0.58 () {56 vars in 1376 bytes} [Thu Aug 25 03:00:42 2022] GET /identity/v3/role_assignments?group.id=f1ea6bd0777d45e896a30daf9eaed344&scope.project.id=b981090d110a4dd99327dbd654b9daa9&role.id=62f19ea272b447ccaab1576cd9bf4fd9 => generated 260 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:00:42.368347 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 190/382] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:00:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 25 03:00:42.869731 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09471607-a177-4fc5-99a5-26b7aad4878a None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:42.987451 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09471607-a177-4fc5-99a5-26b7aad4878a None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:43.002200 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:43.025231 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f210115-f26d-4e84-9f17-0c4fe8db705d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:43.025869 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f210115-f26d-4e84-9f17-0c4fe8db705d None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:43.026150 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f210115-f26d-4e84-9f17-0c4fe8db705d None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:43.026453 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f210115-f26d-4e84-9f17-0c4fe8db705d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:43.040684 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f210115-f26d-4e84-9f17-0c4fe8db705d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:00:43.045394 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-5f210115-f26d-4e84-9f17-0c4fe8db705d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 25 03:00:43.045394 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:00:43.045394 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:00:43.045394 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:00:43.045394 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:00:43.045394 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:00:43.045394 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:00:43.045394 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:00:43.045394 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:00:43.045394 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:00:43.045394 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:00:43.045394 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:00:43.045394 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:00:43.045394 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:00:43.045394 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:00:43.045394 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:00:43.046855 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:00:43.046855 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:00:43.046855 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:00:43.046855 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:00:43.046855 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:00:43.046855 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:00:43.046855 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:00:43.046855 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:00:43.046855 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:00:43.046855 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:00:43.046855 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:00:43.046855 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:00:43.046855 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:00:43.046855 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:00:43.046855 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:00:43.046855 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:00:43.047729 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:00:43.047729 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:00:43.047729 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:00:43.047729 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:00:43.047729 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:00:43.047729 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:00:43.047729 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:00:43.047729 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:00:43.047729 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:00:43.047729 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:00:43.047729 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 25 03:00:43.047729 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:00:43.047729 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 193/383] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Aug 25 03:00:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:00:43.055197 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f59f053-9140-4676-b714-7dfbaadcba9c None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:43.083318 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:43.166185 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f59f053-9140-4676-b714-7dfbaadcba9c None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:43.207840 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7337e939-cd54-4581-a11a-8f77c283bf8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:43.208461 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7337e939-cd54-4581-a11a-8f77c283bf8f None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:43.208742 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7337e939-cd54-4581-a11a-8f77c283bf8f None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:43.209015 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7337e939-cd54-4581-a11a-8f77c283bf8f None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:43.209689 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7337e939-cd54-4581-a11a-8f77c283bf8f None admin] RBAC: Adding query filter params (name=member) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:43.209991 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7337e939-cd54-4581-a11a-8f77c283bf8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:43.211389 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7337e939-cd54-4581-a11a-8f77c283bf8f None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:43.224915 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 191/384] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Aug 25 03:00:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:00:43.231134 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f210115-f26d-4e84-9f17-0c4fe8db705d None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:43.279499 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5868b44-4c58-4969-9e84-087a1d68a958 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:43.279686 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5868b44-4c58-4969-9e84-087a1d68a958 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:43.280612 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5868b44-4c58-4969-9e84-087a1d68a958 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:43.282202 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 192/385] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:00:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:00:43.287385 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d2bbc7-8244-47a3-9429-237cf1f5a9d9 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:43.287668 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d2bbc7-8244-47a3-9429-237cf1f5a9d9 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:43.287942 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d2bbc7-8244-47a3-9429-237cf1f5a9d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:43.311681 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-00d2bbc7-8244-47a3-9429-237cf1f5a9d9 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:00:43.312869 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-00d2bbc7-8244-47a3-9429-237cf1f5a9d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:43.390866 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f210115-f26d-4e84-9f17-0c4fe8db705d None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:43.436562 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e26790-29c9-47f8-8d90-17a6b3c59697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:43.437202 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e26790-29c9-47f8-8d90-17a6b3c59697 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:43.437482 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e26790-29c9-47f8-8d90-17a6b3c59697 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:43.437757 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e26790-29c9-47f8-8d90-17a6b3c59697 None admin] PATH_INFO: `/v3/groups/6e496f4f230c4815b5e45878999d27cf` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:43.442269 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e26790-29c9-47f8-8d90-17a6b3c59697 None admin] RBAC: Authorizing `identity:get_group(group_id=6e496f4f230c4815b5e45878999d27cf)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:43.443752 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e26790-29c9-47f8-8d90-17a6b3c59697 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:43.445340 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 194/386] 10.222.0.58 () {56 vars in 1180 bytes} [Thu Aug 25 03:00:43 2022] GET /identity/v3/groups/6e496f4f230c4815b5e45878999d27cf => generated 226 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:43.447811 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 193/387] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:00:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 25 03:00:43.450964 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e26790-29c9-47f8-8d90-17a6b3c59697 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:43.494653 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:43.561819 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e26790-29c9-47f8-8d90-17a6b3c59697 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:43.603817 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef460911-3820-48d6-abcf-bd592208f7d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:43.604453 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef460911-3820-48d6-abcf-bd592208f7d0 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:43.604734 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef460911-3820-48d6-abcf-bd592208f7d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:43.605008 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef460911-3820-48d6-abcf-bd592208f7d0 None admin] PATH_INFO: `/v3/projects/0d32389226ec4d2dac8c390a33d5436c` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:43.606127 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef460911-3820-48d6-abcf-bd592208f7d0 None admin] RBAC: Authorizing `identity:get_project(project_id=0d32389226ec4d2dac8c390a33d5436c)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:43.607634 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef460911-3820-48d6-abcf-bd592208f7d0 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:43.609656 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 195/388] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Aug 25 03:00:43 2022] GET /identity/v3/projects/0d32389226ec4d2dac8c390a33d5436c => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:43.615064 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d2bbc7-8244-47a3-9429-237cf1f5a9d9 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:43.678691 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:43.752519 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d2bbc7-8244-47a3-9429-237cf1f5a9d9 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:43.787911 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c41b7e3-9169-4bfd-b7e4-20a3ada30e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:43.788584 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c41b7e3-9169-4bfd-b7e4-20a3ada30e00 None admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:43.788860 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c41b7e3-9169-4bfd-b7e4-20a3ada30e00 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:43.789193 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c41b7e3-9169-4bfd-b7e4-20a3ada30e00 None admin] PATH_INFO: `/v3/projects/0d32389226ec4d2dac8c390a33d5436c/groups/6e496f4f230c4815b5e45878999d27cf/roles/3255f77192524a16bc405173106871fc` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:43.791451 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c41b7e3-9169-4bfd-b7e4-20a3ada30e00 None admin] RBAC: Authorizing `identity:create_grant(project_id=0d32389226ec4d2dac8c390a33d5436c, group_id=6e496f4f230c4815b5e45878999d27cf, role_id=3255f77192524a16bc405173106871fc)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:43.793246 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c41b7e3-9169-4bfd-b7e4-20a3ada30e00 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:43.831979 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef460911-3820-48d6-abcf-bd592208f7d0 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:43.881940 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 194/389] 10.222.0.58 () {58 vars in 1442 bytes} [Thu Aug 25 03:00:43 2022] PUT /identity/v3/projects/0d32389226ec4d2dac8c390a33d5436c/groups/6e496f4f230c4815b5e45878999d27cf/roles/3255f77192524a16bc405173106871fc => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:00:43.886541 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc55c9c-ba85-4fdb-bb22-3210869c1de8 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:43.886930 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc55c9c-ba85-4fdb-bb22-3210869c1de8 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:43.887217 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc55c9c-ba85-4fdb-bb22-3210869c1de8 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:43.889013 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 195/390] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:00:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:00:43.907611 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34edc1fb-a631-44f6-83b7-fa9d187d6dfc None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:43.907965 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34edc1fb-a631-44f6-83b7-fa9d187d6dfc None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:43.908270 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34edc1fb-a631-44f6-83b7-fa9d187d6dfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:43.939330 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-34edc1fb-a631-44f6-83b7-fa9d187d6dfc None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:00:43.940546 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-34edc1fb-a631-44f6-83b7-fa9d187d6dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:44.020166 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef460911-3820-48d6-abcf-bd592208f7d0 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:44.042856 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:44.078818 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 196/391] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:00:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 03:00:44.084204 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e0f97a-5dc9-4da2-8233-4f5700a9a3dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:44.085092 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e0f97a-5dc9-4da2-8233-4f5700a9a3dd None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:44.085373 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e0f97a-5dc9-4da2-8233-4f5700a9a3dd None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:44.085645 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e0f97a-5dc9-4da2-8233-4f5700a9a3dd None admin] PATH_INFO: `/v3/roles/member` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:44.097328 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e0f97a-5dc9-4da2-8233-4f5700a9a3dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:00:44.102496 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-95e0f97a-5dc9-4da2-8233-4f5700a9a3dd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 25 03:00:44.102496 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:00:44.102496 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:00:44.102496 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:00:44.102496 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:00:44.102496 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:00:44.102496 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:00:44.102496 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:00:44.102496 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:00:44.102496 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:00:44.102496 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:00:44.102496 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:00:44.102496 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:00:44.102496 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:00:44.102496 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:00:44.102496 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:00:44.104347 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:00:44.104347 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:00:44.104347 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:00:44.104347 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:00:44.104347 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:00:44.104347 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:00:44.104347 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:00:44.104347 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:00:44.104347 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:00:44.104347 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:00:44.104347 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:00:44.104347 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:00:44.104347 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:00:44.104347 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:00:44.104347 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:00:44.104347 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:00:44.105385 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:00:44.105385 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:00:44.105385 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:00:44.105385 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:00:44.105385 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:00:44.105385 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:00:44.105385 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:00:44.105385 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:00:44.105385 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:00:44.105385 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:00:44.105385 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 25 03:00:44.105385 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:00:44.105385 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 196/392] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Aug 25 03:00:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:00:44.114803 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34edc1fb-a631-44f6-83b7-fa9d187d6dfc None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:44.127864 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:44.239927 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34edc1fb-a631-44f6-83b7-fa9d187d6dfc None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:44.281609 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d61b217-52dd-458f-b1e8-0fea4aba115d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:44.282233 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d61b217-52dd-458f-b1e8-0fea4aba115d None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:44.282547 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d61b217-52dd-458f-b1e8-0fea4aba115d None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:44.282820 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d61b217-52dd-458f-b1e8-0fea4aba115d None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:44.283490 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d61b217-52dd-458f-b1e8-0fea4aba115d None admin] RBAC: Adding query filter params (name=member) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:44.283794 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d61b217-52dd-458f-b1e8-0fea4aba115d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:44.285163 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d61b217-52dd-458f-b1e8-0fea4aba115d None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:44.298089 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 197/393] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Aug 25 03:00:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:44.303935 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e0f97a-5dc9-4da2-8233-4f5700a9a3dd None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:44.348069 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6e05c1-7aa2-44ff-a1ad-5b970439ef7e None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:44.348799 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6e05c1-7aa2-44ff-a1ad-5b970439ef7e None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:44.349237 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6e05c1-7aa2-44ff-a1ad-5b970439ef7e None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:44.351616 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 198/394] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:00:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:00:44.358843 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827a95d6-68e1-4dd6-bd26-528a88e08e0c None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:44.359313 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827a95d6-68e1-4dd6-bd26-528a88e08e0c None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:44.359755 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827a95d6-68e1-4dd6-bd26-528a88e08e0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:44.381333 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-827a95d6-68e1-4dd6-bd26-528a88e08e0c None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:00:44.383348 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-827a95d6-68e1-4dd6-bd26-528a88e08e0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:44.438965 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e0f97a-5dc9-4da2-8233-4f5700a9a3dd None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:44.466427 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 199/395] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:00:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 03:00:44.467326 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827a95d6-68e1-4dd6-bd26-528a88e08e0c None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:44.474623 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:44.482054 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d72d8a-00b7-4f40-b5fc-d51db996719a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:44.482916 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d72d8a-00b7-4f40-b5fc-d51db996719a None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:44.483864 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d72d8a-00b7-4f40-b5fc-d51db996719a None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:44.484303 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d72d8a-00b7-4f40-b5fc-d51db996719a None admin] PATH_INFO: `/v3/groups/6e496f4f230c4815b5e45878999d27cf` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:44.485578 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d72d8a-00b7-4f40-b5fc-d51db996719a None admin] RBAC: Authorizing `identity:get_group(group_id=6e496f4f230c4815b5e45878999d27cf)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:44.487273 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d72d8a-00b7-4f40-b5fc-d51db996719a None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:44.489652 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 197/396] 10.222.0.58 () {56 vars in 1180 bytes} [Thu Aug 25 03:00:44 2022] GET /identity/v3/groups/6e496f4f230c4815b5e45878999d27cf => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:44.495534 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d72d8a-00b7-4f40-b5fc-d51db996719a None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:44.571673 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:44.611838 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827a95d6-68e1-4dd6-bd26-528a88e08e0c None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:44.659420 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d72d8a-00b7-4f40-b5fc-d51db996719a None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:44.664172 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3888bb1d-0fa4-4eaf-a3c2-3c2e241fdd2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:44.665287 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3888bb1d-0fa4-4eaf-a3c2-3c2e241fdd2a None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:44.670204 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3888bb1d-0fa4-4eaf-a3c2-3c2e241fdd2a None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:44.671399 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3888bb1d-0fa4-4eaf-a3c2-3c2e241fdd2a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:44.687902 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3888bb1d-0fa4-4eaf-a3c2-3c2e241fdd2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:00:44.693737 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-3888bb1d-0fa4-4eaf-a3c2-3c2e241fdd2a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 25 03:00:44.693737 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:00:44.693737 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:00:44.693737 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:00:44.693737 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:00:44.693737 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:00:44.693737 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:00:44.693737 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:00:44.693737 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:00:44.693737 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:00:44.693737 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:00:44.693737 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:00:44.693737 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:00:44.693737 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:00:44.693737 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:00:44.693737 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:00:44.694651 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:00:44.694651 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:00:44.694651 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:00:44.694651 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:00:44.694651 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:00:44.694651 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:00:44.694651 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:00:44.694651 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:00:44.694651 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:00:44.694651 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:00:44.694651 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:00:44.694651 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:00:44.694651 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:00:44.694651 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:00:44.694651 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:00:44.694651 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:00:44.695440 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:00:44.695440 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:00:44.695440 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:00:44.695440 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:00:44.695440 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:00:44.695440 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:00:44.695440 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:00:44.695440 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:00:44.695440 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:00:44.695440 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:00:44.695440 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 25 03:00:44.695440 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:00:44.703018 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 200/397] 10.222.0.58 () {56 vars in 1114 bytes} [Thu Aug 25 03:00:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:00:44.709201 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3888bb1d-0fa4-4eaf-a3c2-3c2e241fdd2a None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:44.726546 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf99c6d-c544-4f7b-a9c2-cb056e57fec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:44.727508 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf99c6d-c544-4f7b-a9c2-cb056e57fec8 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:44.729208 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf99c6d-c544-4f7b-a9c2-cb056e57fec8 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:44.729700 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf99c6d-c544-4f7b-a9c2-cb056e57fec8 None admin] PATH_INFO: `/v3/projects/54952dee2ab9420d9541e99a1825f5af` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:44.732204 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf99c6d-c544-4f7b-a9c2-cb056e57fec8 None admin] RBAC: Authorizing `identity:get_project(project_id=54952dee2ab9420d9541e99a1825f5af)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:44.734434 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf99c6d-c544-4f7b-a9c2-cb056e57fec8 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:44.742863 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 198/398] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Aug 25 03:00:44 2022] GET /identity/v3/projects/54952dee2ab9420d9541e99a1825f5af => generated 302 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:44.746120 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf99c6d-c544-4f7b-a9c2-cb056e57fec8 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:44.825133 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:44.856253 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf99c6d-c544-4f7b-a9c2-cb056e57fec8 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:44.860565 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:44.878973 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3888bb1d-0fa4-4eaf-a3c2-3c2e241fdd2a None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:44.907661 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b852fe-26e2-46c7-9225-dd01c1f36a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:44.908773 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b852fe-26e2-46c7-9225-dd01c1f36a0e None admin] REQUEST_METHOD: `PUT` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:44.909713 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b852fe-26e2-46c7-9225-dd01c1f36a0e None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:44.910497 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b852fe-26e2-46c7-9225-dd01c1f36a0e None admin] PATH_INFO: `/v3/projects/54952dee2ab9420d9541e99a1825f5af/groups/6e496f4f230c4815b5e45878999d27cf/roles/3255f77192524a16bc405173106871fc` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:44.913668 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b852fe-26e2-46c7-9225-dd01c1f36a0e None admin] RBAC: Authorizing `identity:create_grant(project_id=54952dee2ab9420d9541e99a1825f5af, group_id=6e496f4f230c4815b5e45878999d27cf, role_id=3255f77192524a16bc405173106871fc)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:44.916168 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b852fe-26e2-46c7-9225-dd01c1f36a0e None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:44.921360 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd9bed2-08b7-4f27-b25a-7824d9826280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:44.921904 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd9bed2-08b7-4f27-b25a-7824d9826280 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:44.922185 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd9bed2-08b7-4f27-b25a-7824d9826280 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:44.922583 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd9bed2-08b7-4f27-b25a-7824d9826280 None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:44.923277 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd9bed2-08b7-4f27-b25a-7824d9826280 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:44.923584 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd9bed2-08b7-4f27-b25a-7824d9826280 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:44.925061 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd9bed2-08b7-4f27-b25a-7824d9826280 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:44.933800 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 201/399] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Aug 25 03:00:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:44.938423 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd9bed2-08b7-4f27-b25a-7824d9826280 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:44.987285 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 199/400] 10.222.0.58 () {58 vars in 1442 bytes} [Thu Aug 25 03:00:44 2022] PUT /identity/v3/projects/54952dee2ab9420d9541e99a1825f5af/groups/6e496f4f230c4815b5e45878999d27cf/roles/3255f77192524a16bc405173106871fc => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:00:44.988118 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b852fe-26e2-46c7-9225-dd01c1f36a0e None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:45.065343 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd9bed2-08b7-4f27-b25a-7824d9826280 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:45.099903 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:45.110055 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b852fe-26e2-46c7-9225-dd01c1f36a0e None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:45.115343 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2f979f-a454-4bd6-a80a-dd4694986cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:45.116003 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2f979f-a454-4bd6-a80a-dd4694986cfb None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:45.116286 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2f979f-a454-4bd6-a80a-dd4694986cfb None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:45.116559 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2f979f-a454-4bd6-a80a-dd4694986cfb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:45.132044 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2f979f-a454-4bd6-a80a-dd4694986cfb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:00:45.139213 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-dd2f979f-a454-4bd6-a80a-dd4694986cfb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 25 03:00:45.139213 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:00:45.139213 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:00:45.139213 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:00:45.139213 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:00:45.139213 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:00:45.139213 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:00:45.139213 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:00:45.139213 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:00:45.139213 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:00:45.139213 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:00:45.139213 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:00:45.139213 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:00:45.139213 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:00:45.139213 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:00:45.139213 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:00:45.140168 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:00:45.140168 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:00:45.140168 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:00:45.140168 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:00:45.140168 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:00:45.140168 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:00:45.140168 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:00:45.140168 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:00:45.140168 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:00:45.140168 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:00:45.140168 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:00:45.140168 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:00:45.140168 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:00:45.140168 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:00:45.140168 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:00:45.140168 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:00:45.140952 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:00:45.140952 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:00:45.140952 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:00:45.140952 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:00:45.140952 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:00:45.140952 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:00:45.140952 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:00:45.140952 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:00:45.140952 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:00:45.140952 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:00:45.140952 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 25 03:00:45.140952 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:00:45.140952 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 202/401] 10.222.0.58 () {56 vars in 1114 bytes} [Thu Aug 25 03:00:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:00:45.142134 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6ea5db-28a3-4b30-8edf-098d003feb8d None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:45.143326 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6ea5db-28a3-4b30-8edf-098d003feb8d None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:45.143326 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6ea5db-28a3-4b30-8edf-098d003feb8d None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:45.144166 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 203/402] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:00:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:00:45.147166 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6ea5db-28a3-4b30-8edf-098d003feb8d None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:45.161907 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb926fe-ef2c-400e-b5e9-cdeadabf85d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:45.162592 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb926fe-ef2c-400e-b5e9-cdeadabf85d0 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:45.163167 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb926fe-ef2c-400e-b5e9-cdeadabf85d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:45.163457 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb926fe-ef2c-400e-b5e9-cdeadabf85d0 None admin] PATH_INFO: `/v3/groups/6e496f4f230c4815b5e45878999d27cf` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:45.164632 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb926fe-ef2c-400e-b5e9-cdeadabf85d0 None admin] RBAC: Authorizing `identity:get_group(group_id=6e496f4f230c4815b5e45878999d27cf)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:45.166365 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb926fe-ef2c-400e-b5e9-cdeadabf85d0 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:45.168535 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 200/403] 10.222.0.58 () {56 vars in 1180 bytes} [Thu Aug 25 03:00:44 2022] GET /identity/v3/groups/6e496f4f230c4815b5e45878999d27cf => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:45.169549 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d07da0-0813-4239-a329-367abc1e3a62 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:45.169970 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d07da0-0813-4239-a329-367abc1e3a62 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:45.170095 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d07da0-0813-4239-a329-367abc1e3a62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:45.199429 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-38d07da0-0813-4239-a329-367abc1e3a62 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:00:45.200959 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-38d07da0-0813-4239-a329-367abc1e3a62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:45.227964 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:45.272721 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6ea5db-28a3-4b30-8edf-098d003feb8d None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:45.275539 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 201/404] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:00:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 03:00:45.276484 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d07da0-0813-4239-a329-367abc1e3a62 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:45.338123 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444ba0bd-410f-4caf-9069-84a4620f6009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:45.338761 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444ba0bd-410f-4caf-9069-84a4620f6009 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:45.339043 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444ba0bd-410f-4caf-9069-84a4620f6009 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:45.339317 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444ba0bd-410f-4caf-9069-84a4620f6009 None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:45.339981 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444ba0bd-410f-4caf-9069-84a4620f6009 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:45.340283 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444ba0bd-410f-4caf-9069-84a4620f6009 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:45.341658 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444ba0bd-410f-4caf-9069-84a4620f6009 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:45.361765 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 204/405] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Aug 25 03:00:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:45.368617 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444ba0bd-410f-4caf-9069-84a4620f6009 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:45.429828 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:45.444440 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d07da0-0813-4239-a329-367abc1e3a62 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:45.482396 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6982b853-6b27-4c17-889c-7fdfcb1c3eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:45.483140 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6982b853-6b27-4c17-889c-7fdfcb1c3eb6 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:45.483920 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6982b853-6b27-4c17-889c-7fdfcb1c3eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:45.484362 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6982b853-6b27-4c17-889c-7fdfcb1c3eb6 None admin] PATH_INFO: `/v3/projects/0d32389226ec4d2dac8c390a33d5436c` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:45.486221 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6982b853-6b27-4c17-889c-7fdfcb1c3eb6 None admin] RBAC: Authorizing `identity:get_project(project_id=0d32389226ec4d2dac8c390a33d5436c)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:45.487789 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6982b853-6b27-4c17-889c-7fdfcb1c3eb6 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:45.487981 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444ba0bd-410f-4caf-9069-84a4620f6009 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:45.489923 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 202/406] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Aug 25 03:00:45 2022] GET /identity/v3/projects/0d32389226ec4d2dac8c390a33d5436c => generated 298 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:45.495013 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6982b853-6b27-4c17-889c-7fdfcb1c3eb6 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:45.533420 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:45.539310 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb99530-ec22-496b-acf7-f3da3035ae15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:45.540398 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb99530-ec22-496b-acf7-f3da3035ae15 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:45.543145 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb99530-ec22-496b-acf7-f3da3035ae15 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:45.543145 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb99530-ec22-496b-acf7-f3da3035ae15 None admin] PATH_INFO: `/v3/groups/6e496f4f230c4815b5e45878999d27cf` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:45.543145 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb99530-ec22-496b-acf7-f3da3035ae15 None admin] RBAC: Authorizing `identity:get_group(group_id=6e496f4f230c4815b5e45878999d27cf)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:45.544267 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb99530-ec22-496b-acf7-f3da3035ae15 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:45.546772 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 205/407] 10.222.0.58 () {56 vars in 1180 bytes} [Thu Aug 25 03:00:45 2022] GET /identity/v3/groups/6e496f4f230c4815b5e45878999d27cf => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:00:45.555264 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb99530-ec22-496b-acf7-f3da3035ae15 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:45.651131 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6982b853-6b27-4c17-889c-7fdfcb1c3eb6 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:45.697937 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cf9348-961a-4a3b-bdb4-93e8627c6011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:45.698546 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cf9348-961a-4a3b-bdb4-93e8627c6011 None admin] REQUEST_METHOD: `PUT` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:45.698825 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cf9348-961a-4a3b-bdb4-93e8627c6011 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:45.699094 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cf9348-961a-4a3b-bdb4-93e8627c6011 None admin] PATH_INFO: `/v3/projects/0d32389226ec4d2dac8c390a33d5436c/groups/6e496f4f230c4815b5e45878999d27cf/roles/7886cc7e333a42d08085932494a3ea6b` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:45.701359 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cf9348-961a-4a3b-bdb4-93e8627c6011 None admin] RBAC: Authorizing `identity:create_grant(project_id=0d32389226ec4d2dac8c390a33d5436c, group_id=6e496f4f230c4815b5e45878999d27cf, role_id=7886cc7e333a42d08085932494a3ea6b)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:45.703117 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cf9348-961a-4a3b-bdb4-93e8627c6011 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:45.703842 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb99530-ec22-496b-acf7-f3da3035ae15 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:45.718504 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:45.739091 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26bc9b8-811d-4255-8c5b-f7221c951f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:45.739725 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26bc9b8-811d-4255-8c5b-f7221c951f3e None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:45.740004 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26bc9b8-811d-4255-8c5b-f7221c951f3e None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:45.740275 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26bc9b8-811d-4255-8c5b-f7221c951f3e None admin] PATH_INFO: `/v3/projects/54952dee2ab9420d9541e99a1825f5af` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:45.741381 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26bc9b8-811d-4255-8c5b-f7221c951f3e None admin] RBAC: Authorizing `identity:get_project(project_id=54952dee2ab9420d9541e99a1825f5af)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:45.743011 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26bc9b8-811d-4255-8c5b-f7221c951f3e None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:45.744887 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 206/408] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Aug 25 03:00:45 2022] GET /identity/v3/projects/54952dee2ab9420d9541e99a1825f5af => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:45.745435 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26bc9b8-811d-4255-8c5b-f7221c951f3e None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:45.806414 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 203/409] 10.222.0.58 () {58 vars in 1442 bytes} [Thu Aug 25 03:00:45 2022] PUT /identity/v3/projects/0d32389226ec4d2dac8c390a33d5436c/groups/6e496f4f230c4815b5e45878999d27cf/roles/7886cc7e333a42d08085932494a3ea6b => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:00:45.807082 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cf9348-961a-4a3b-bdb4-93e8627c6011 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:45.923618 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26bc9b8-811d-4255-8c5b-f7221c951f3e None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:45.988912 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1363a95-abc6-487a-a8ba-8b09bf0ef3af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:45.989533 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1363a95-abc6-487a-a8ba-8b09bf0ef3af None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:45.989814 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1363a95-abc6-487a-a8ba-8b09bf0ef3af None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:45.990087 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1363a95-abc6-487a-a8ba-8b09bf0ef3af None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:45.995540 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:46.003982 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1363a95-abc6-487a-a8ba-8b09bf0ef3af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:00:46.007458 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cf9348-961a-4a3b-bdb4-93e8627c6011 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:46.011237 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-d1363a95-abc6-487a-a8ba-8b09bf0ef3af None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 03:00:46.011237 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:00:46.011237 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:00:46.011237 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:00:46.011237 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:00:46.011237 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:00:46.011237 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:00:46.011237 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:00:46.011237 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:00:46.011237 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:00:46.011237 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:00:46.011237 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:00:46.011237 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:00:46.011237 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:00:46.011237 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:00:46.011237 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:00:46.012127 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:00:46.012127 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:00:46.012127 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:00:46.012127 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:00:46.012127 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:00:46.012127 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:00:46.012127 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:00:46.012127 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:00:46.012127 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:00:46.012127 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:00:46.012127 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:00:46.012127 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:00:46.012127 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:00:46.012127 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:00:46.012127 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:00:46.012127 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:00:46.012890 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:00:46.012890 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:00:46.012890 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:00:46.012890 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:00:46.012890 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:00:46.012890 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:00:46.012890 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:00:46.012890 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:00:46.012890 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:00:46.012890 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:00:46.012890 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 03:00:46.012890 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:00:46.020013 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 207/410] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Aug 25 03:00:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:00:46.021742 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1363a95-abc6-487a-a8ba-8b09bf0ef3af None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:46.058122 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05927437-ac05-480d-acd9-bff9fe239589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:46.059230 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05927437-ac05-480d-acd9-bff9fe239589 None admin] REQUEST_METHOD: `PUT` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:46.059526 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05927437-ac05-480d-acd9-bff9fe239589 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:46.059805 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05927437-ac05-480d-acd9-bff9fe239589 None admin] PATH_INFO: `/v3/projects/54952dee2ab9420d9541e99a1825f5af/groups/6e496f4f230c4815b5e45878999d27cf/roles/7886cc7e333a42d08085932494a3ea6b` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:46.062490 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05927437-ac05-480d-acd9-bff9fe239589 None admin] RBAC: Authorizing `identity:create_grant(project_id=54952dee2ab9420d9541e99a1825f5af, group_id=6e496f4f230c4815b5e45878999d27cf, role_id=7886cc7e333a42d08085932494a3ea6b)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:46.064514 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05927437-ac05-480d-acd9-bff9fe239589 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:46.080119 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:46.142568 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1363a95-abc6-487a-a8ba-8b09bf0ef3af None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:46.143544 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 204/411] 10.222.0.58 () {58 vars in 1442 bytes} [Thu Aug 25 03:00:45 2022] PUT /identity/v3/projects/54952dee2ab9420d9541e99a1825f5af/groups/6e496f4f230c4815b5e45878999d27cf/roles/7886cc7e333a42d08085932494a3ea6b => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:00:46.184174 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246adcf9-fdc0-45db-a834-184a10dea02d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:46.185207 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246adcf9-fdc0-45db-a834-184a10dea02d None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:46.185730 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246adcf9-fdc0-45db-a834-184a10dea02d None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:46.186430 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246adcf9-fdc0-45db-a834-184a10dea02d None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:46.187329 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246adcf9-fdc0-45db-a834-184a10dea02d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:46.187834 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246adcf9-fdc0-45db-a834-184a10dea02d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:46.189456 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246adcf9-fdc0-45db-a834-184a10dea02d None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:46.204356 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 208/412] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Aug 25 03:00:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:00:46.206378 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05927437-ac05-480d-acd9-bff9fe239589 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:46.256169 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2491a503-3683-4b80-961d-905f10a8aade None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:46.256816 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2491a503-3683-4b80-961d-905f10a8aade None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:46.257291 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2491a503-3683-4b80-961d-905f10a8aade None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:46.259201 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 209/413] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:00:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:00:46.266832 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a41dee-f8d8-4c16-a764-0191509974c1 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:46.267113 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a41dee-f8d8-4c16-a764-0191509974c1 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:46.267396 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a41dee-f8d8-4c16-a764-0191509974c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:46.287208 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-08a41dee-f8d8-4c16-a764-0191509974c1 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:00:46.288421 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-08a41dee-f8d8-4c16-a764-0191509974c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:46.326139 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05927437-ac05-480d-acd9-bff9fe239589 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:46.383951 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 210/414] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:00:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 25 03:00:46.388812 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc330ea-449f-40e6-82c6-dc28a4ffad22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:46.390074 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc330ea-449f-40e6-82c6-dc28a4ffad22 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:46.390459 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc330ea-449f-40e6-82c6-dc28a4ffad22 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:46.390809 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc330ea-449f-40e6-82c6-dc28a4ffad22 None admin] PATH_INFO: `/v3/groups/f1ea6bd0777d45e896a30daf9eaed344` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:46.392050 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc330ea-449f-40e6-82c6-dc28a4ffad22 None admin] RBAC: Authorizing `identity:get_group(group_id=f1ea6bd0777d45e896a30daf9eaed344)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:46.393586 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc330ea-449f-40e6-82c6-dc28a4ffad22 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:46.395662 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 205/415] 10.222.0.58 () {56 vars in 1180 bytes} [Thu Aug 25 03:00:46 2022] GET /identity/v3/groups/f1ea6bd0777d45e896a30daf9eaed344 => generated 229 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:46.401152 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a41dee-f8d8-4c16-a764-0191509974c1 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:46.442642 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:46.506101 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a41dee-f8d8-4c16-a764-0191509974c1 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:46.545952 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfa16d1-5a07-4a1c-b2ae-9236ed9870b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:46.546344 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfa16d1-5a07-4a1c-b2ae-9236ed9870b1 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:46.546628 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfa16d1-5a07-4a1c-b2ae-9236ed9870b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:46.546904 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfa16d1-5a07-4a1c-b2ae-9236ed9870b1 None admin] PATH_INFO: `/v3/projects/b981090d110a4dd99327dbd654b9daa9` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:46.550946 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfa16d1-5a07-4a1c-b2ae-9236ed9870b1 None admin] RBAC: Authorizing `identity:get_project(project_id=b981090d110a4dd99327dbd654b9daa9)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:46.552507 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfa16d1-5a07-4a1c-b2ae-9236ed9870b1 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:46.554569 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 211/416] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Aug 25 03:00:46 2022] GET /identity/v3/projects/b981090d110a4dd99327dbd654b9daa9 => generated 344 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:00:46.560270 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc330ea-449f-40e6-82c6-dc28a4ffad22 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:46.584048 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:46.677006 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfa16d1-5a07-4a1c-b2ae-9236ed9870b1 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:46.696074 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc330ea-449f-40e6-82c6-dc28a4ffad22 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:46.761236 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e538b320-16c3-4478-9672-56458026c63c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:46.761907 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e538b320-16c3-4478-9672-56458026c63c None admin] REQUEST_METHOD: `PUT` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:46.762189 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e538b320-16c3-4478-9672-56458026c63c None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:46.762488 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e538b320-16c3-4478-9672-56458026c63c None admin] PATH_INFO: `/v3/projects/b981090d110a4dd99327dbd654b9daa9/groups/f1ea6bd0777d45e896a30daf9eaed344/roles/62f19ea272b447ccaab1576cd9bf4fd9` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:46.766635 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e538b320-16c3-4478-9672-56458026c63c None admin] RBAC: Authorizing `identity:create_grant(project_id=b981090d110a4dd99327dbd654b9daa9, group_id=f1ea6bd0777d45e896a30daf9eaed344, role_id=62f19ea272b447ccaab1576cd9bf4fd9)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:46.768108 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e538b320-16c3-4478-9672-56458026c63c None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:46.809764 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfa16d1-5a07-4a1c-b2ae-9236ed9870b1 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:46.822627 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:46.838016 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 206/417] 10.222.0.58 () {58 vars in 1442 bytes} [Thu Aug 25 03:00:46 2022] PUT /identity/v3/projects/b981090d110a4dd99327dbd654b9daa9/groups/f1ea6bd0777d45e896a30daf9eaed344/roles/62f19ea272b447ccaab1576cd9bf4fd9 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:00:46.869201 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc111c81-13b4-453b-a72e-4f038c598dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:46.869872 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc111c81-13b4-453b-a72e-4f038c598dd0 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:46.870663 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc111c81-13b4-453b-a72e-4f038c598dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:46.870936 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc111c81-13b4-453b-a72e-4f038c598dd0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:46.892910 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc111c81-13b4-453b-a72e-4f038c598dd0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:00:46.897864 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-bc111c81-13b4-453b-a72e-4f038c598dd0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 25 03:00:46.897864 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:00:46.897864 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:00:46.897864 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:00:46.897864 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:00:46.897864 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:00:46.897864 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:00:46.897864 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:00:46.897864 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:00:46.897864 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:00:46.897864 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:00:46.897864 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:00:46.897864 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:00:46.897864 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:00:46.897864 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:00:46.897864 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:00:46.898876 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:00:46.898876 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:00:46.898876 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:00:46.898876 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:00:46.898876 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:00:46.898876 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:00:46.898876 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:00:46.898876 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:00:46.898876 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:00:46.898876 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:00:46.898876 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:00:46.898876 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:00:46.898876 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:00:46.898876 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:00:46.898876 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:00:46.898876 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:00:46.899676 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:00:46.899676 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:00:46.899676 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:00:46.899676 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:00:46.899676 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:00:46.899676 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:00:46.899676 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:00:46.899676 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:00:46.899676 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:00:46.899676 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:00:46.899676 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 25 03:00:46.899676 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:00:46.899676 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 212/418] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Aug 25 03:00:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 03:00:46.905151 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e538b320-16c3-4478-9672-56458026c63c None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:46.993092 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:47.015876 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e538b320-16c3-4478-9672-56458026c63c None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:47.073380 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd12cd91-e283-4bdc-96cc-a91ff4201259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:47.073921 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd12cd91-e283-4bdc-96cc-a91ff4201259 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:47.074204 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd12cd91-e283-4bdc-96cc-a91ff4201259 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:47.074512 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd12cd91-e283-4bdc-96cc-a91ff4201259 None admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:47.075191 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd12cd91-e283-4bdc-96cc-a91ff4201259 None admin] RBAC: Adding query filter params (name=member) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:47.075493 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd12cd91-e283-4bdc-96cc-a91ff4201259 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:47.077508 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd12cd91-e283-4bdc-96cc-a91ff4201259 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:47.091387 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 207/419] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Aug 25 03:00:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:47.096863 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc111c81-13b4-453b-a72e-4f038c598dd0 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:47.217996 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc111c81-13b4-453b-a72e-4f038c598dd0 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:47.272042 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d360952-6e8c-4a26-a86a-ae24058b3731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:47.272659 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d360952-6e8c-4a26-a86a-ae24058b3731 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:47.272942 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d360952-6e8c-4a26-a86a-ae24058b3731 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:47.273218 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d360952-6e8c-4a26-a86a-ae24058b3731 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:47.292058 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6ce18a-cced-4cb5-a3b0-bcef675b8e9b None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:47.292333 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6ce18a-cced-4cb5-a3b0-bcef675b8e9b None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:47.293289 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6ce18a-cced-4cb5-a3b0-bcef675b8e9b None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:47.294088 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 208/420] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:00:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:00:47.305326 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65159c2-0654-44f2-b34d-34dc9220b18f None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:47.305617 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65159c2-0654-44f2-b34d-34dc9220b18f None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:47.305877 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65159c2-0654-44f2-b34d-34dc9220b18f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:47.314953 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:47.348179 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-d65159c2-0654-44f2-b34d-34dc9220b18f None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:00:47.349317 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-d65159c2-0654-44f2-b34d-34dc9220b18f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:47.354667 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d360952-6e8c-4a26-a86a-ae24058b3731 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:47.356371 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d360952-6e8c-4a26-a86a-ae24058b3731 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:47.467602 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 209/421] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:00:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 03:00:47.521189 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 213/422] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Aug 25 03:00:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 25 03:00:47.527127 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65159c2-0654-44f2-b34d-34dc9220b18f None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:47.626753 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:47.669532 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65159c2-0654-44f2-b34d-34dc9220b18f None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:47.718209 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd104fe2-3b08-49df-b3a8-777b49066a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:47.718883 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd104fe2-3b08-49df-b3a8-777b49066a1d None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:47.719160 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd104fe2-3b08-49df-b3a8-777b49066a1d None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:47.719438 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd104fe2-3b08-49df-b3a8-777b49066a1d None admin] PATH_INFO: `/v3/projects/0d32389226ec4d2dac8c390a33d5436c` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:47.723102 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd104fe2-3b08-49df-b3a8-777b49066a1d None admin] RBAC: Authorizing `identity:get_project(project_id=0d32389226ec4d2dac8c390a33d5436c)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:47.724618 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd104fe2-3b08-49df-b3a8-777b49066a1d None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:47.726636 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 210/423] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Aug 25 03:00:47 2022] GET /identity/v3/projects/0d32389226ec4d2dac8c390a33d5436c => generated 298 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:47.731916 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d360952-6e8c-4a26-a86a-ae24058b3731 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:47.824334 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:47.878563 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d360952-6e8c-4a26-a86a-ae24058b3731 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:47.913567 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168d55bc-7da4-4121-a4fe-727c665a239d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:47.914592 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168d55bc-7da4-4121-a4fe-727c665a239d None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:47.915077 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168d55bc-7da4-4121-a4fe-727c665a239d None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:47.915565 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168d55bc-7da4-4121-a4fe-727c665a239d None admin] PATH_INFO: `/v3/groups/6e496f4f230c4815b5e45878999d27cf` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:47.916949 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168d55bc-7da4-4121-a4fe-727c665a239d None admin] RBAC: Authorizing `identity:get_group(group_id=6e496f4f230c4815b5e45878999d27cf)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:47.918638 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168d55bc-7da4-4121-a4fe-727c665a239d None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:47.920844 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 214/424] 10.222.0.58 () {56 vars in 1180 bytes} [Thu Aug 25 03:00:47 2022] GET /identity/v3/groups/6e496f4f230c4815b5e45878999d27cf => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:47.926475 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd104fe2-3b08-49df-b3a8-777b49066a1d None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:47.970037 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168d55bc-7da4-4121-a4fe-727c665a239d None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:48.052576 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd104fe2-3b08-49df-b3a8-777b49066a1d None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:48.089460 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cd7a47-db21-4112-8d5d-5cc3b1aa4554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:48.090355 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cd7a47-db21-4112-8d5d-5cc3b1aa4554 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:48.090885 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cd7a47-db21-4112-8d5d-5cc3b1aa4554 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:48.091334 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cd7a47-db21-4112-8d5d-5cc3b1aa4554 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:48.092206 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cd7a47-db21-4112-8d5d-5cc3b1aa4554 None admin] RBAC: Adding query filter params (group.id=6e496f4f230c4815b5e45878999d27cf, role.id=3255f77192524a16bc405173106871fc, scope.project.id=0d32389226ec4d2dac8c390a33d5436c) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:48.093220 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cd7a47-db21-4112-8d5d-5cc3b1aa4554 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:48.094832 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cd7a47-db21-4112-8d5d-5cc3b1aa4554 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:48.101410 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 211/425] 10.222.0.58 () {56 vars in 1376 bytes} [Thu Aug 25 03:00:47 2022] GET /identity/v3/role_assignments?group.id=6e496f4f230c4815b5e45878999d27cf&scope.project.id=0d32389226ec4d2dac8c390a33d5436c&role.id=3255f77192524a16bc405173106871fc => generated 611 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:00:48.197292 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80db70f-63a1-42de-b4bc-c1c9937905b6 None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:48.198554 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80db70f-63a1-42de-b4bc-c1c9937905b6 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:48.199044 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80db70f-63a1-42de-b4bc-c1c9937905b6 None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:48.201382 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 212/426] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:00:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 25 03:00:48.209410 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a2b335-d630-42be-9d9f-b8c1b9df1ca0 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:48.210007 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a2b335-d630-42be-9d9f-b8c1b9df1ca0 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:48.210632 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a2b335-d630-42be-9d9f-b8c1b9df1ca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:48.219186 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168d55bc-7da4-4121-a4fe-727c665a239d None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:48.226673 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:48.239675 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-e0a2b335-d630-42be-9d9f-b8c1b9df1ca0 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:00:48.240930 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-e0a2b335-d630-42be-9d9f-b8c1b9df1ca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:48.281778 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:48.295629 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf5ecc4-ef28-446c-9f0d-5268153d0d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:48.296292 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf5ecc4-ef28-446c-9f0d-5268153d0d9d None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:48.296699 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf5ecc4-ef28-446c-9f0d-5268153d0d9d None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:48.296979 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf5ecc4-ef28-446c-9f0d-5268153d0d9d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:48.318812 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf5ecc4-ef28-446c-9f0d-5268153d0d9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:00:48.326425 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-bcf5ecc4-ef28-446c-9f0d-5268153d0d9d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 25 03:00:48.326425 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:00:48.326425 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:00:48.326425 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:00:48.326425 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:00:48.326425 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:00:48.326425 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:00:48.326425 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:00:48.326425 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:00:48.326425 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:00:48.326425 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:00:48.326425 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:00:48.326425 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:00:48.326425 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:00:48.326425 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:00:48.326425 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:00:48.327880 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:00:48.327880 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:00:48.327880 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:00:48.327880 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:00:48.327880 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:00:48.327880 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:00:48.327880 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:00:48.327880 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:00:48.327880 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:00:48.327880 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:00:48.327880 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:00:48.327880 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:00:48.327880 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:00:48.327880 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:00:48.327880 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:00:48.327880 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:00:48.328670 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:00:48.328670 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:00:48.328670 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:00:48.328670 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:00:48.328670 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:00:48.328670 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:00:48.328670 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:00:48.328670 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:00:48.328670 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:00:48.328670 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:00:48.328670 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 25 03:00:48.328670 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:00:48.328670 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 215/427] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Aug 25 03:00:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:00:48.334042 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf5ecc4-ef28-446c-9f0d-5268153d0d9d None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:48.367258 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 213/428] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:00:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 03:00:48.447242 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf5ecc4-ef28-446c-9f0d-5268153d0d9d None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:48.483023 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5efbf2-057d-48e5-8520-b896b95a5dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:48.483601 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5efbf2-057d-48e5-8520-b896b95a5dd4 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:48.483876 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5efbf2-057d-48e5-8520-b896b95a5dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:48.484154 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5efbf2-057d-48e5-8520-b896b95a5dd4 None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:48.484853 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5efbf2-057d-48e5-8520-b896b95a5dd4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:48.485131 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5efbf2-057d-48e5-8520-b896b95a5dd4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:48.486532 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5efbf2-057d-48e5-8520-b896b95a5dd4 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:48.495246 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 216/429] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Aug 25 03:00:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:48.501332 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a2b335-d630-42be-9d9f-b8c1b9df1ca0 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:48.573406 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202b9ac8-2964-4dd4-a74f-80d22617a058 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:48.573716 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202b9ac8-2964-4dd4-a74f-80d22617a058 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:48.573973 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202b9ac8-2964-4dd4-a74f-80d22617a058 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:48.575682 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 217/430] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:00:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 25 03:00:48.592078 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c1d0e4-a739-4963-81a6-4072ad6daf77 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:48.592539 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c1d0e4-a739-4963-81a6-4072ad6daf77 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:48.592925 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c1d0e4-a739-4963-81a6-4072ad6daf77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:48.614774 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-a2c1d0e4-a739-4963-81a6-4072ad6daf77 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:00:48.616199 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-a2c1d0e4-a739-4963-81a6-4072ad6daf77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:48.616842 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a2b335-d630-42be-9d9f-b8c1b9df1ca0 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:48.654237 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:48.664209 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d111670a-2256-42a0-a1f9-461447e15645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:48.665076 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d111670a-2256-42a0-a1f9-461447e15645 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:48.665495 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d111670a-2256-42a0-a1f9-461447e15645 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:48.665926 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d111670a-2256-42a0-a1f9-461447e15645 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:48.677642 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:48.700982 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 218/431] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:00:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 25 03:00:48.701985 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c1d0e4-a739-4963-81a6-4072ad6daf77 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:48.764141 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d111670a-2256-42a0-a1f9-461447e15645 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:48.766407 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d111670a-2256-42a0-a1f9-461447e15645 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:48.851593 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c1d0e4-a739-4963-81a6-4072ad6daf77 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:48.900162 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de80efbf-7e93-4527-978e-f0c4ee34a0bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:48.901106 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de80efbf-7e93-4527-978e-f0c4ee34a0bc None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:48.901595 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de80efbf-7e93-4527-978e-f0c4ee34a0bc None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:48.902065 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de80efbf-7e93-4527-978e-f0c4ee34a0bc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:48.905796 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 214/432] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Aug 25 03:00:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 25 03:00:48.910749 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d111670a-2256-42a0-a1f9-461447e15645 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:48.915010 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de80efbf-7e93-4527-978e-f0c4ee34a0bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:00:48.922575 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-de80efbf-7e93-4527-978e-f0c4ee34a0bc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 25 03:00:48.922575 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:00:48.922575 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:00:48.922575 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:00:48.922575 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:00:48.922575 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:00:48.922575 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:00:48.922575 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:00:48.922575 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:00:48.922575 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:00:48.922575 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:00:48.922575 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:00:48.922575 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:00:48.922575 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:00:48.922575 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:00:48.922575 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:00:48.923473 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:00:48.923473 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:00:48.923473 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:00:48.923473 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:00:48.923473 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:00:48.923473 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:00:48.923473 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:00:48.923473 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:00:48.923473 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:00:48.923473 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:00:48.923473 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:00:48.923473 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:00:48.923473 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:00:48.923473 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:00:48.923473 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:00:48.923473 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:00:48.924264 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:00:48.924264 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:00:48.924264 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:00:48.924264 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:00:48.924264 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:00:48.924264 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:00:48.924264 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:00:48.924264 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:00:48.924264 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:00:48.924264 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:00:48.924264 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 25 03:00:48.924264 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:00:48.926986 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 219/433] 10.222.0.58 () {56 vars in 1114 bytes} [Thu Aug 25 03:00:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:00:48.932735 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de80efbf-7e93-4527-978e-f0c4ee34a0bc None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:48.954389 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:48.995953 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:49.064852 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d111670a-2256-42a0-a1f9-461447e15645 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:49.067388 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de80efbf-7e93-4527-978e-f0c4ee34a0bc None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:49.106186 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71de27e9-0494-42de-ad42-c2110f7515c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:49.107056 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71de27e9-0494-42de-ad42-c2110f7515c0 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:49.107470 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71de27e9-0494-42de-ad42-c2110f7515c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:49.107924 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658a9103-5151-4feb-b1fb-65188e28ae0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:49.108255 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71de27e9-0494-42de-ad42-c2110f7515c0 None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:49.109130 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658a9103-5151-4feb-b1fb-65188e28ae0d None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:49.109409 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658a9103-5151-4feb-b1fb-65188e28ae0d None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:49.109681 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658a9103-5151-4feb-b1fb-65188e28ae0d None admin] PATH_INFO: `/v3/projects/54952dee2ab9420d9541e99a1825f5af` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:49.110036 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71de27e9-0494-42de-ad42-c2110f7515c0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:49.110507 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71de27e9-0494-42de-ad42-c2110f7515c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:49.110827 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658a9103-5151-4feb-b1fb-65188e28ae0d None admin] RBAC: Authorizing `identity:get_project(project_id=54952dee2ab9420d9541e99a1825f5af)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:49.112246 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658a9103-5151-4feb-b1fb-65188e28ae0d None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:49.112366 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71de27e9-0494-42de-ad42-c2110f7515c0 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:49.118151 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 215/434] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Aug 25 03:00:48 2022] GET /identity/v3/projects/54952dee2ab9420d9541e99a1825f5af => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:49.118151 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658a9103-5151-4feb-b1fb-65188e28ae0d None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:49.126368 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 220/435] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Aug 25 03:00:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:49.130578 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71de27e9-0494-42de-ad42-c2110f7515c0 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:49.216389 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:49.220038 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658a9103-5151-4feb-b1fb-65188e28ae0d None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:49.238103 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71de27e9-0494-42de-ad42-c2110f7515c0 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:49.240940 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:49.257733 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7debab-df28-4717-9ac2-ebb9ef8cf57f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:49.258609 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7debab-df28-4717-9ac2-ebb9ef8cf57f None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:49.258895 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7debab-df28-4717-9ac2-ebb9ef8cf57f None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:49.259170 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7debab-df28-4717-9ac2-ebb9ef8cf57f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:49.270587 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7debab-df28-4717-9ac2-ebb9ef8cf57f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:00:49.274829 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f546a462-3fcd-4174-972a-23bf8993f8da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:49.275370 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-2d7debab-df28-4717-9ac2-ebb9ef8cf57f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 25 03:00:49.275370 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:00:49.275370 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:00:49.275370 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:00:49.275370 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:00:49.275370 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:00:49.275370 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:00:49.275370 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:00:49.275370 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:00:49.275370 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:00:49.275370 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:00:49.275370 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:00:49.275370 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:00:49.275370 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:00:49.275370 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:00:49.275370 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:00:49.276238 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:00:49.276238 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:00:49.276238 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:00:49.276238 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:00:49.276238 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:00:49.276238 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:00:49.276238 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:00:49.276238 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:00:49.276238 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:00:49.276238 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:00:49.276238 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:00:49.276238 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:00:49.276238 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:00:49.276238 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:00:49.276238 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:00:49.276238 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:00:49.277038 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:00:49.277038 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:00:49.277038 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:00:49.277038 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:00:49.277038 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:00:49.277038 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:00:49.277038 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:00:49.277038 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:00:49.277038 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:00:49.277038 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:00:49.277038 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 25 03:00:49.277038 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:00:49.278149 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f546a462-3fcd-4174-972a-23bf8993f8da None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:49.278149 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f546a462-3fcd-4174-972a-23bf8993f8da None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:49.278149 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f546a462-3fcd-4174-972a-23bf8993f8da None admin] PATH_INFO: `/v3/groups/6e496f4f230c4815b5e45878999d27cf` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:49.278149 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f546a462-3fcd-4174-972a-23bf8993f8da None admin] RBAC: Authorizing `identity:get_group(group_id=6e496f4f230c4815b5e45878999d27cf)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:49.278572 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f546a462-3fcd-4174-972a-23bf8993f8da None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:49.285528 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 221/436] 10.222.0.58 () {56 vars in 1180 bytes} [Thu Aug 25 03:00:49 2022] GET /identity/v3/groups/6e496f4f230c4815b5e45878999d27cf => generated 226 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:49.285665 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 216/437] 10.222.0.58 () {56 vars in 1114 bytes} [Thu Aug 25 03:00:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:00:49.285755 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f546a462-3fcd-4174-972a-23bf8993f8da None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:49.285838 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7debab-df28-4717-9ac2-ebb9ef8cf57f None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:49.392916 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f546a462-3fcd-4174-972a-23bf8993f8da None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:49.397503 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7debab-df28-4717-9ac2-ebb9ef8cf57f None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:49.425942 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79b5edb-d85b-4f4e-aaca-28562ced2bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:49.426574 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79b5edb-d85b-4f4e-aaca-28562ced2bae None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:49.426850 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79b5edb-d85b-4f4e-aaca-28562ced2bae None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:49.427122 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79b5edb-d85b-4f4e-aaca-28562ced2bae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:49.430879 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b1871f-b337-4ded-9035-6100b09812d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:49.431478 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b1871f-b337-4ded-9035-6100b09812d7 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:49.431757 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b1871f-b337-4ded-9035-6100b09812d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:49.432031 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b1871f-b337-4ded-9035-6100b09812d7 None admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:49.432701 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b1871f-b337-4ded-9035-6100b09812d7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:49.433006 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b1871f-b337-4ded-9035-6100b09812d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:49.434403 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b1871f-b337-4ded-9035-6100b09812d7 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:49.446584 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 217/438] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Aug 25 03:00:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:49.446584 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b1871f-b337-4ded-9035-6100b09812d7 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:49.466514 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:49.467381 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:49.492630 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79b5edb-d85b-4f4e-aaca-28562ced2bae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:49.494113 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79b5edb-d85b-4f4e-aaca-28562ced2bae None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:49.541509 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b1871f-b337-4ded-9035-6100b09812d7 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:49.575141 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcbe8ea-1cfb-468b-a6a1-d21d2344a5f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:49.575743 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcbe8ea-1cfb-468b-a6a1-d21d2344a5f9 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:49.576025 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcbe8ea-1cfb-468b-a6a1-d21d2344a5f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:49.576298 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcbe8ea-1cfb-468b-a6a1-d21d2344a5f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:49.577798 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcbe8ea-1cfb-468b-a6a1-d21d2344a5f9 None admin] RBAC: Adding query filter params (group.id=6e496f4f230c4815b5e45878999d27cf, role.id=3255f77192524a16bc405173106871fc, scope.project.id=54952dee2ab9420d9541e99a1825f5af) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:49.578109 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcbe8ea-1cfb-468b-a6a1-d21d2344a5f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:49.579614 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcbe8ea-1cfb-468b-a6a1-d21d2344a5f9 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:49.585509 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 218/439] 10.222.0.58 () {56 vars in 1376 bytes} [Thu Aug 25 03:00:49 2022] GET /identity/v3/role_assignments?group.id=6e496f4f230c4815b5e45878999d27cf&scope.project.id=54952dee2ab9420d9541e99a1825f5af&role.id=3255f77192524a16bc405173106871fc => generated 611 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:49.586705 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edcef07-8d84-4b21-a007-894570bf6fba None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:49.586993 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edcef07-8d84-4b21-a007-894570bf6fba None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:49.587267 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edcef07-8d84-4b21-a007-894570bf6fba None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:49.588603 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 219/440] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:00:49 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:00:49.589193 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edcef07-8d84-4b21-a007-894570bf6fba None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:49.598093 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 222/441] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Aug 25 03:00:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 25 03:00:49.599464 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b0894e-8417-48d4-8284-ba4e8124389f None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:49.600918 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b0894e-8417-48d4-8284-ba4e8124389f None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:49.600918 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b0894e-8417-48d4-8284-ba4e8124389f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:49.620013 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-b8b0894e-8417-48d4-8284-ba4e8124389f None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:00:49.621331 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-b8b0894e-8417-48d4-8284-ba4e8124389f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:49.692378 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 223/442] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:00:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 03:00:49.700286 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:49.700975 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b0894e-8417-48d4-8284-ba4e8124389f None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:49.707517 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:49.729748 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edcef07-8d84-4b21-a007-894570bf6fba None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:49.771306 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7a465c-cc6c-4ec6-853f-e1dd0830eab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:49.772298 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7a465c-cc6c-4ec6-853f-e1dd0830eab5 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:49.772783 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7a465c-cc6c-4ec6-853f-e1dd0830eab5 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:49.773271 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7a465c-cc6c-4ec6-853f-e1dd0830eab5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:49.823738 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b0894e-8417-48d4-8284-ba4e8124389f None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:49.847545 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7a465c-cc6c-4ec6-853f-e1dd0830eab5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:49.849019 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7a465c-cc6c-4ec6-853f-e1dd0830eab5 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:49.860655 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f3ebc7-2dfe-46b2-9c37-4f1f52c40a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:49.861286 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f3ebc7-2dfe-46b2-9c37-4f1f52c40a8d None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:49.861569 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f3ebc7-2dfe-46b2-9c37-4f1f52c40a8d None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:49.861842 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f3ebc7-2dfe-46b2-9c37-4f1f52c40a8d None admin] PATH_INFO: `/v3/projects/0d32389226ec4d2dac8c390a33d5436c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:49.862961 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f3ebc7-2dfe-46b2-9c37-4f1f52c40a8d None admin] RBAC: Authorizing `identity:get_project(project_id=0d32389226ec4d2dac8c390a33d5436c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:49.864394 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f3ebc7-2dfe-46b2-9c37-4f1f52c40a8d None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:49.865993 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 224/443] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Aug 25 03:00:49 2022] GET /identity/v3/projects/0d32389226ec4d2dac8c390a33d5436c => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:49.871411 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f3ebc7-2dfe-46b2-9c37-4f1f52c40a8d None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:49.940062 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:49.940992 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:49.956563 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 220/444] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Aug 25 03:00:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 25 03:00:49.962379 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7a465c-cc6c-4ec6-853f-e1dd0830eab5 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:49.978130 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f3ebc7-2dfe-46b2-9c37-4f1f52c40a8d None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:50.011922 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eebef08-5508-4768-8da7-a951998c30cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:50.012549 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eebef08-5508-4768-8da7-a951998c30cb None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:50.012832 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eebef08-5508-4768-8da7-a951998c30cb None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:50.013108 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eebef08-5508-4768-8da7-a951998c30cb None admin] PATH_INFO: `/v3/groups/6e496f4f230c4815b5e45878999d27cf` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:50.014247 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eebef08-5508-4768-8da7-a951998c30cb None admin] RBAC: Authorizing `identity:get_group(group_id=6e496f4f230c4815b5e45878999d27cf)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:50.015732 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eebef08-5508-4768-8da7-a951998c30cb None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:50.017300 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 225/445] 10.222.0.58 () {56 vars in 1180 bytes} [Thu Aug 25 03:00:49 2022] GET /identity/v3/groups/6e496f4f230c4815b5e45878999d27cf => generated 226 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:00:50.017300 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eebef08-5508-4768-8da7-a951998c30cb None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:50.058971 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7a465c-cc6c-4ec6-853f-e1dd0830eab5 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:50.091548 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ee3212-10cc-4293-80ac-22833f90c206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:50.092177 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ee3212-10cc-4293-80ac-22833f90c206 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:50.092959 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ee3212-10cc-4293-80ac-22833f90c206 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:50.093258 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ee3212-10cc-4293-80ac-22833f90c206 None admin] PATH_INFO: `/v3/projects/54952dee2ab9420d9541e99a1825f5af` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:50.094410 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ee3212-10cc-4293-80ac-22833f90c206 None admin] RBAC: Authorizing `identity:get_project(project_id=54952dee2ab9420d9541e99a1825f5af)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:50.095853 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ee3212-10cc-4293-80ac-22833f90c206 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:50.097463 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 221/446] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Aug 25 03:00:49 2022] GET /identity/v3/projects/54952dee2ab9420d9541e99a1825f5af => generated 302 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:00:50.098220 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ee3212-10cc-4293-80ac-22833f90c206 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:50.118347 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eebef08-5508-4768-8da7-a951998c30cb None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:50.151080 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ef7a80-b883-4959-92ba-bc6337ff82ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:50.151699 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ef7a80-b883-4959-92ba-bc6337ff82ad None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:50.151972 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ef7a80-b883-4959-92ba-bc6337ff82ad None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:50.152243 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ef7a80-b883-4959-92ba-bc6337ff82ad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:50.152624 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:50.159454 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:50.163762 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ef7a80-b883-4959-92ba-bc6337ff82ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:00:50.168354 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-24ef7a80-b883-4959-92ba-bc6337ff82ad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 03:00:50.168354 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:00:50.168354 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:00:50.168354 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:00:50.168354 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:00:50.168354 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:00:50.168354 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:00:50.168354 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:00:50.168354 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:00:50.168354 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:00:50.168354 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:00:50.168354 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:00:50.168354 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:00:50.168354 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:00:50.168354 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:00:50.168354 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:00:50.169240 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:00:50.169240 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:00:50.169240 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:00:50.169240 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:00:50.169240 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:00:50.169240 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:00:50.169240 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:00:50.169240 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:00:50.169240 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:00:50.169240 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:00:50.169240 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:00:50.169240 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:00:50.169240 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:00:50.169240 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:00:50.169240 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:00:50.169240 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:00:50.170051 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:00:50.170051 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:00:50.170051 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:00:50.170051 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:00:50.170051 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:00:50.170051 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:00:50.170051 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:00:50.170051 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:00:50.170051 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:00:50.170051 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:00:50.170051 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 03:00:50.170051 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:00:50.170051 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 226/447] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Aug 25 03:00:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:00:50.170855 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ef7a80-b883-4959-92ba-bc6337ff82ad None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:50.196563 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ee3212-10cc-4293-80ac-22833f90c206 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:50.230903 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c074e6d-8567-4d1e-983b-6b0d18dfb824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:50.231450 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c074e6d-8567-4d1e-983b-6b0d18dfb824 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:50.231731 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c074e6d-8567-4d1e-983b-6b0d18dfb824 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:50.232008 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c074e6d-8567-4d1e-983b-6b0d18dfb824 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:50.232725 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c074e6d-8567-4d1e-983b-6b0d18dfb824 None admin] RBAC: Adding query filter params (group.id=6e496f4f230c4815b5e45878999d27cf, role.id=7886cc7e333a42d08085932494a3ea6b, scope.project.id=0d32389226ec4d2dac8c390a33d5436c) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:50.233036 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c074e6d-8567-4d1e-983b-6b0d18dfb824 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:50.234553 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c074e6d-8567-4d1e-983b-6b0d18dfb824 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:50.240128 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 222/448] 10.222.0.58 () {56 vars in 1376 bytes} [Thu Aug 25 03:00:50 2022] GET /identity/v3/role_assignments?group.id=6e496f4f230c4815b5e45878999d27cf&scope.project.id=0d32389226ec4d2dac8c390a33d5436c&role.id=7886cc7e333a42d08085932494a3ea6b => generated 611 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:50.240901 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c074e6d-8567-4d1e-983b-6b0d18dfb824 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:50.274480 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ef7a80-b883-4959-92ba-bc6337ff82ad None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:50.310457 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faf57f8-304a-47cb-b23d-e68ec597922a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:50.311103 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faf57f8-304a-47cb-b23d-e68ec597922a None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:50.311381 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faf57f8-304a-47cb-b23d-e68ec597922a None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:50.311653 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faf57f8-304a-47cb-b23d-e68ec597922a None admin] PATH_INFO: `/v3/groups/6e496f4f230c4815b5e45878999d27cf` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:50.312822 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9faf57f8-304a-47cb-b23d-e68ec597922a None admin] RBAC: Authorizing `identity:get_group(group_id=6e496f4f230c4815b5e45878999d27cf)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:50.314323 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9faf57f8-304a-47cb-b23d-e68ec597922a None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:50.321123 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 227/449] 10.222.0.58 () {56 vars in 1180 bytes} [Thu Aug 25 03:00:50 2022] GET /identity/v3/groups/6e496f4f230c4815b5e45878999d27cf => generated 226 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:50.321881 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faf57f8-304a-47cb-b23d-e68ec597922a None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:50.351290 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c074e6d-8567-4d1e-983b-6b0d18dfb824 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:50.370077 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:50.394464 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24ba1c4-8f56-4c24-9ad5-1583e0db4ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:50.395091 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24ba1c4-8f56-4c24-9ad5-1583e0db4ac7 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:50.395382 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24ba1c4-8f56-4c24-9ad5-1583e0db4ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:50.395660 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24ba1c4-8f56-4c24-9ad5-1583e0db4ac7 None admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:50.396326 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24ba1c4-8f56-4c24-9ad5-1583e0db4ac7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:50.396633 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24ba1c4-8f56-4c24-9ad5-1583e0db4ac7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:50.398018 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24ba1c4-8f56-4c24-9ad5-1583e0db4ac7 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:50.401555 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:50.411129 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 223/450] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Aug 25 03:00:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:50.416681 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24ba1c4-8f56-4c24-9ad5-1583e0db4ac7 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:50.446212 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faf57f8-304a-47cb-b23d-e68ec597922a None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:50.485578 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cbc6ab-ab52-4065-80d0-fc4d01c10d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:50.486114 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cbc6ab-ab52-4065-80d0-fc4d01c10d58 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:50.486492 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cbc6ab-ab52-4065-80d0-fc4d01c10d58 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:50.486780 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cbc6ab-ab52-4065-80d0-fc4d01c10d58 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:50.487495 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cbc6ab-ab52-4065-80d0-fc4d01c10d58 None admin] RBAC: Adding query filter params (group.id=6e496f4f230c4815b5e45878999d27cf, role.id=7886cc7e333a42d08085932494a3ea6b, scope.project.id=54952dee2ab9420d9541e99a1825f5af) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:50.487795 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cbc6ab-ab52-4065-80d0-fc4d01c10d58 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:50.489222 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cbc6ab-ab52-4065-80d0-fc4d01c10d58 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:50.495980 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 228/451] 10.222.0.58 () {56 vars in 1376 bytes} [Thu Aug 25 03:00:50 2022] GET /identity/v3/role_assignments?group.id=6e496f4f230c4815b5e45878999d27cf&scope.project.id=54952dee2ab9420d9541e99a1825f5af&role.id=7886cc7e333a42d08085932494a3ea6b => generated 611 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:50.536887 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24ba1c4-8f56-4c24-9ad5-1583e0db4ac7 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:50.576699 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6aa33f-a3b9-4122-8390-32bd1d18dd04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:50.577885 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6aa33f-a3b9-4122-8390-32bd1d18dd04 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:50.578369 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6aa33f-a3b9-4122-8390-32bd1d18dd04 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:50.578792 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6aa33f-a3b9-4122-8390-32bd1d18dd04 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:50.637125 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:50.658549 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6aa33f-a3b9-4122-8390-32bd1d18dd04 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:50.660287 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6aa33f-a3b9-4122-8390-32bd1d18dd04 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:50.759814 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 224/452] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Aug 25 03:00:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 25 03:00:50.767025 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cbc6ab-ab52-4065-80d0-fc4d01c10d58 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:50.869402 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cbc6ab-ab52-4065-80d0-fc4d01c10d58 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:50.898671 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:50.903516 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d6e845-4c02-47b9-92b4-209a9cb97c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:50.904153 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d6e845-4c02-47b9-92b4-209a9cb97c7c None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:50.904431 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d6e845-4c02-47b9-92b4-209a9cb97c7c None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:50.904843 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d6e845-4c02-47b9-92b4-209a9cb97c7c None admin] PATH_INFO: `/v3/projects/b981090d110a4dd99327dbd654b9daa9` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:50.906549 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d6e845-4c02-47b9-92b4-209a9cb97c7c None admin] RBAC: Authorizing `identity:get_project(project_id=b981090d110a4dd99327dbd654b9daa9)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:50.907972 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d6e845-4c02-47b9-92b4-209a9cb97c7c None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:50.914774 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 229/453] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Aug 25 03:00:50 2022] GET /identity/v3/projects/b981090d110a4dd99327dbd654b9daa9 => generated 344 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:50.915158 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6aa33f-a3b9-4122-8390-32bd1d18dd04 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:50.993554 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:51.011697 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6aa33f-a3b9-4122-8390-32bd1d18dd04 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:51.045146 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801ab685-f35e-4415-9bb9-a53d209803aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:51.045812 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801ab685-f35e-4415-9bb9-a53d209803aa None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:51.046090 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801ab685-f35e-4415-9bb9-a53d209803aa None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:51.046390 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801ab685-f35e-4415-9bb9-a53d209803aa None admin] PATH_INFO: `/v3/groups/f1ea6bd0777d45e896a30daf9eaed344` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:51.047458 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801ab685-f35e-4415-9bb9-a53d209803aa None admin] RBAC: Authorizing `identity:get_group(group_id=f1ea6bd0777d45e896a30daf9eaed344)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:51.048899 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801ab685-f35e-4415-9bb9-a53d209803aa None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:51.050448 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 225/454] 10.222.0.58 () {56 vars in 1180 bytes} [Thu Aug 25 03:00:50 2022] GET /identity/v3/groups/f1ea6bd0777d45e896a30daf9eaed344 => generated 229 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:51.056362 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d6e845-4c02-47b9-92b4-209a9cb97c7c None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:51.151430 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d6e845-4c02-47b9-92b4-209a9cb97c7c None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:51.182898 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dfd3d2-0deb-4455-9b0b-0fb3f5726a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:51.183484 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dfd3d2-0deb-4455-9b0b-0fb3f5726a54 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:51.183758 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dfd3d2-0deb-4455-9b0b-0fb3f5726a54 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:51.184028 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dfd3d2-0deb-4455-9b0b-0fb3f5726a54 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:51.184740 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dfd3d2-0deb-4455-9b0b-0fb3f5726a54 None admin] RBAC: Adding query filter params (group.id=f1ea6bd0777d45e896a30daf9eaed344, role.id=62f19ea272b447ccaab1576cd9bf4fd9, scope.project.id=b981090d110a4dd99327dbd654b9daa9) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:51.185043 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dfd3d2-0deb-4455-9b0b-0fb3f5726a54 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:51.186646 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dfd3d2-0deb-4455-9b0b-0fb3f5726a54 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:51.192101 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 230/455] 10.222.0.58 () {56 vars in 1376 bytes} [Thu Aug 25 03:00:51 2022] GET /identity/v3/role_assignments?group.id=f1ea6bd0777d45e896a30daf9eaed344&scope.project.id=b981090d110a4dd99327dbd654b9daa9&role.id=62f19ea272b447ccaab1576cd9bf4fd9 => generated 611 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:53.446627 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf3a165-7431-410f-9f5b-6149dee57ccb None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:53.446627 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf3a165-7431-410f-9f5b-6149dee57ccb None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:53.446627 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf3a165-7431-410f-9f5b-6149dee57ccb None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:53.446627 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 226/456] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:00:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:00:53.452672 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b0818c-20ad-4864-a37c-4f24fcc174c8 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:53.453483 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b0818c-20ad-4864-a37c-4f24fcc174c8 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:53.453953 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b0818c-20ad-4864-a37c-4f24fcc174c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:53.478193 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-f1b0818c-20ad-4864-a37c-4f24fcc174c8 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:00:53.479733 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-f1b0818c-20ad-4864-a37c-4f24fcc174c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:53.554819 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:53.568876 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 231/457] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:00:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 03:00:53.636680 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc2e5ea-9086-4ec4-b21b-d63109b62412 None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:53.637180 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc2e5ea-9086-4ec4-b21b-d63109b62412 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:53.637672 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc2e5ea-9086-4ec4-b21b-d63109b62412 None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:53.639218 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 227/458] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:00:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:00:53.644648 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9004e0-a349-4573-a311-9252a35a3038 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:53.645171 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9004e0-a349-4573-a311-9252a35a3038 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:53.645659 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9004e0-a349-4573-a311-9252a35a3038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:53.667656 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-da9004e0-a349-4573-a311-9252a35a3038 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:00:53.669159 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-da9004e0-a349-4573-a311-9252a35a3038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:53.760798 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 232/459] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:00:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 03:00:53.974634 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc2e5ea-9086-4ec4-b21b-d63109b62412 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:54.074749 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9004e0-a349-4573-a311-9252a35a3038 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:54.114646 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc2e5ea-9086-4ec4-b21b-d63109b62412 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:54.159227 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3589e838-af92-4d00-9739-e44d36447b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:54.159872 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3589e838-af92-4d00-9739-e44d36447b96 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:54.160156 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3589e838-af92-4d00-9739-e44d36447b96 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:54.160891 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3589e838-af92-4d00-9739-e44d36447b96 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:54.176320 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3589e838-af92-4d00-9739-e44d36447b96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:54.178107 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3589e838-af92-4d00-9739-e44d36447b96 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:54.189618 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9004e0-a349-4573-a311-9252a35a3038 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:54.190360 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:54.198339 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-3589e838-af92-4d00-9739-e44d36447b96 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:00:54.198339 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:00:54.198339 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:00:54.198339 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:00:54.198339 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:00:54.198339 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:00:54.198339 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:00:54.198339 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:00:54.198339 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:00:54.198339 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:00:54.198339 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:00:54.198339 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:00:54.198339 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 03:00:54.198339 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 03:00:54.198339 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 03:00:54.198339 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 03:00:54.199246 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:00:54.199246 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:00:54.199246 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:00:54.199246 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:00:54.199246 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:00:54.199246 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:00:54.199246 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:00:54.199246 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:00:54.199246 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:00:54.199246 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:00:54.199246 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:00:54.199246 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:00:54.199246 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:00:54.199246 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:00:54.199246 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:00:54.199246 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:00:54.199246 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 03:00:54.200076 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 03:00:54.200076 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:00:54.200076 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:00:54.202558 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 228/460] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:00:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:00:54.203944 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e77cf07-3d65-4fbb-b782-d39d0cf4fe08 None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:54.204437 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e77cf07-3d65-4fbb-b782-d39d0cf4fe08 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:54.210212 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e77cf07-3d65-4fbb-b782-d39d0cf4fe08 None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:54.210212 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 229/461] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:00:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:00:54.211630 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e77cf07-3d65-4fbb-b782-d39d0cf4fe08 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:54.218574 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:54.234509 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a976da13-a3d7-4c8c-82ec-a2390135b12c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:54.235155 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a976da13-a3d7-4c8c-82ec-a2390135b12c None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:54.235440 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a976da13-a3d7-4c8c-82ec-a2390135b12c None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:54.235715 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a976da13-a3d7-4c8c-82ec-a2390135b12c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:54.250453 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a976da13-a3d7-4c8c-82ec-a2390135b12c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:54.251910 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a976da13-a3d7-4c8c-82ec-a2390135b12c None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:54.277014 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-a976da13-a3d7-4c8c-82ec-a2390135b12c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:00:54.277014 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:00:54.277014 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:00:54.277014 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:00:54.277014 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:00:54.277014 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:00:54.277014 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:00:54.277014 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:00:54.277014 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:00:54.277014 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:00:54.277014 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:00:54.277014 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:00:54.277014 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 03:00:54.277014 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 03:00:54.277014 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 03:00:54.277014 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 03:00:54.277969 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:00:54.277969 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:00:54.277969 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:00:54.277969 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:00:54.277969 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:00:54.277969 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:00:54.277969 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:00:54.277969 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:00:54.277969 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:00:54.277969 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:00:54.277969 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:00:54.277969 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:00:54.277969 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:00:54.277969 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:00:54.277969 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:00:54.277969 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:00:54.277969 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 03:00:54.278826 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 03:00:54.278826 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:00:54.278826 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:00:54.278826 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 233/462] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:00:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 03:00:54.284569 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304c3cc9-f12c-4b73-bff7-3cd5e0cd9983 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:54.284569 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304c3cc9-f12c-4b73-bff7-3cd5e0cd9983 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:54.284569 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304c3cc9-f12c-4b73-bff7-3cd5e0cd9983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:54.302236 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-304c3cc9-f12c-4b73-bff7-3cd5e0cd9983 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:00:54.303463 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-304c3cc9-f12c-4b73-bff7-3cd5e0cd9983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:54.334635 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e77cf07-3d65-4fbb-b782-d39d0cf4fe08 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:54.370608 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dddb3a-6d39-416d-bb8b-1d4e8d993ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:54.371215 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dddb3a-6d39-416d-bb8b-1d4e8d993ed4 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:54.371514 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dddb3a-6d39-416d-bb8b-1d4e8d993ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:54.371766 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dddb3a-6d39-416d-bb8b-1d4e8d993ed4 None admin] PATH_INFO: `/v3/domains` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:54.372412 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dddb3a-6d39-416d-bb8b-1d4e8d993ed4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:54.382500 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dddb3a-6d39-416d-bb8b-1d4e8d993ed4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:54.382626 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 234/463] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:00:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 03:00:54.382626 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304c3cc9-f12c-4b73-bff7-3cd5e0cd9983 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:54.382765 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dddb3a-6d39-416d-bb8b-1d4e8d993ed4 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:54.404581 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 230/464] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Aug 25 03:00:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:54.406668 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026716e7-1190-470e-8be4-2648c54a1e49 None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:54.406958 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026716e7-1190-470e-8be4-2648c54a1e49 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:54.407232 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026716e7-1190-470e-8be4-2648c54a1e49 None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:54.408572 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 231/465] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:00:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:00:54.409252 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026716e7-1190-470e-8be4-2648c54a1e49 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:54.474220 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:54.503754 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304c3cc9-f12c-4b73-bff7-3cd5e0cd9983 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:54.506518 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:54.519730 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026716e7-1190-470e-8be4-2648c54a1e49 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:54.551136 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ef7109-9164-4ad4-95ae-228ecb2e7087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:54.551988 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ef7109-9164-4ad4-95ae-228ecb2e7087 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:54.552473 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ef7109-9164-4ad4-95ae-228ecb2e7087 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:54.552824 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ef7109-9164-4ad4-95ae-228ecb2e7087 None admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:54.553844 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ef7109-9164-4ad4-95ae-228ecb2e7087 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:54.554231 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ef7109-9164-4ad4-95ae-228ecb2e7087 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:54.556614 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ef7109-9164-4ad4-95ae-228ecb2e7087 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:54.559884 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3a1557-a6cb-4482-810c-bc952c627333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:54.560657 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3a1557-a6cb-4482-810c-bc952c627333 None admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:54.561083 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3a1557-a6cb-4482-810c-bc952c627333 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:54.561546 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3a1557-a6cb-4482-810c-bc952c627333 None admin] PATH_INFO: `/v3/users` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:54.562357 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3a1557-a6cb-4482-810c-bc952c627333 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:54.563920 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3a1557-a6cb-4482-810c-bc952c627333 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:54.571059 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 235/466] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Aug 25 03:00:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:54.572750 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0094f8a4-d469-496e-866c-269c8358b9aa None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:54.573677 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0094f8a4-d469-496e-866c-269c8358b9aa None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:54.574086 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0094f8a4-d469-496e-866c-269c8358b9aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:54.595810 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-0094f8a4-d469-496e-866c-269c8358b9aa None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:00:54.596932 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-0094f8a4-d469-496e-866c-269c8358b9aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:54.673194 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 232/468] 10.222.0.58 () {60 vars in 1132 bytes} [Thu Aug 25 03:00:54 2022] POST /identity/v3/users => generated 246 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:00:54.673418 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 236/468] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:00:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 03:00:54.678620 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0094f8a4-d469-496e-866c-269c8358b9aa None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:54.693978 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3a1557-a6cb-4482-810c-bc952c627333 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:54.799962 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:54.807737 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0094f8a4-d469-496e-866c-269c8358b9aa None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:54.850739 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68254c8b-c782-427e-93f3-84b54acc9ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:54.851276 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68254c8b-c782-427e-93f3-84b54acc9ff6 None admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:54.851559 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68254c8b-c782-427e-93f3-84b54acc9ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:54.851830 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68254c8b-c782-427e-93f3-84b54acc9ff6 None admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:54.852477 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68254c8b-c782-427e-93f3-84b54acc9ff6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:54.854075 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68254c8b-c782-427e-93f3-84b54acc9ff6 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:54.912759 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3a1557-a6cb-4482-810c-bc952c627333 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:54.931361 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:54.945700 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 237/469] 10.222.0.58 () {60 vars in 1132 bytes} [Thu Aug 25 03:00:54 2022] POST /identity/v3/users => generated 248 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:00:54.955781 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363e72fa-f46a-476f-ac14-7f68227cc6c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:54.956747 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363e72fa-f46a-476f-ac14-7f68227cc6c3 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:54.959600 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363e72fa-f46a-476f-ac14-7f68227cc6c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:54.959600 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363e72fa-f46a-476f-ac14-7f68227cc6c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:54.960584 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68254c8b-c782-427e-93f3-84b54acc9ff6 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:54.972969 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363e72fa-f46a-476f-ac14-7f68227cc6c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:54.975843 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363e72fa-f46a-476f-ac14-7f68227cc6c3 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:54.999399 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-363e72fa-f46a-476f-ac14-7f68227cc6c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:00:54.999399 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:00:54.999399 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:00:54.999399 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:00:54.999399 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:00:54.999399 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:00:54.999399 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:00:54.999399 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:00:54.999399 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:00:54.999399 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:00:54.999399 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:00:54.999399 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:00:54.999399 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 03:00:54.999399 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 03:00:54.999399 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 03:00:54.999399 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 03:00:55.000349 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:00:55.000349 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:00:55.000349 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:00:55.000349 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:00:55.000349 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:00:55.000349 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:00:55.000349 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:00:55.000349 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:00:55.000349 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:00:55.000349 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:00:55.000349 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:00:55.000349 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:00:55.000349 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:00:55.000349 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:00:55.000349 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:00:55.000349 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:00:55.000349 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 03:00:55.001216 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 03:00:55.001216 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:00:55.001216 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:00:55.004067 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 233/470] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:00:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:00:55.010072 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363e72fa-f46a-476f-ac14-7f68227cc6c3 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:55.103780 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68254c8b-c782-427e-93f3-84b54acc9ff6 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:55.161374 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c103162-31f4-4a62-921a-bf77244a37e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:55.162019 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c103162-31f4-4a62-921a-bf77244a37e0 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:55.162253 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363e72fa-f46a-476f-ac14-7f68227cc6c3 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:55.162389 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c103162-31f4-4a62-921a-bf77244a37e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:55.162586 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c103162-31f4-4a62-921a-bf77244a37e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:55.176636 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c103162-31f4-4a62-921a-bf77244a37e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:55.178690 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c103162-31f4-4a62-921a-bf77244a37e0 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:55.195711 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-2c103162-31f4-4a62-921a-bf77244a37e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:00:55.195711 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:00:55.195711 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:00:55.195711 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:00:55.195711 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:00:55.195711 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:00:55.195711 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:00:55.195711 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:00:55.195711 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:00:55.195711 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:00:55.195711 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:00:55.195711 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:00:55.195711 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 03:00:55.195711 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 03:00:55.195711 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 03:00:55.195711 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 03:00:55.196635 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:00:55.196635 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:00:55.196635 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:00:55.196635 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:00:55.196635 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:00:55.196635 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:00:55.196635 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:00:55.196635 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:00:55.196635 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:00:55.196635 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:00:55.196635 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:00:55.196635 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:00:55.196635 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:00:55.196635 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:00:55.196635 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:00:55.196635 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:00:55.196635 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 03:00:55.197460 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 03:00:55.197460 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:00:55.197460 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:00:55.197460 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 238/471] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:00:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:00:55.198232 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6987ff53-e423-401e-8bf7-a28ee4087c78 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:55.198232 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6987ff53-e423-401e-8bf7-a28ee4087c78 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:55.199550 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6987ff53-e423-401e-8bf7-a28ee4087c78 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:55.202609 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 239/472] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:00:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:00:55.204872 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6987ff53-e423-401e-8bf7-a28ee4087c78 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:55.206198 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2186f46-49e7-47f1-9a85-66d0ebd14101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:55.206867 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2186f46-49e7-47f1-9a85-66d0ebd14101 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:55.207153 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2186f46-49e7-47f1-9a85-66d0ebd14101 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:55.207428 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2186f46-49e7-47f1-9a85-66d0ebd14101 None admin] PATH_INFO: `/v3/domains` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:55.208096 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2186f46-49e7-47f1-9a85-66d0ebd14101 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:55.208201 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:55.208391 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2186f46-49e7-47f1-9a85-66d0ebd14101 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:55.209786 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2186f46-49e7-47f1-9a85-66d0ebd14101 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:55.223342 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 234/473] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Aug 25 03:00:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:55.226370 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ce90d3-fabb-4cc1-bed6-e5648e0ef5ce None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:55.226370 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ce90d3-fabb-4cc1-bed6-e5648e0ef5ce None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:55.226370 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ce90d3-fabb-4cc1-bed6-e5648e0ef5ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:55.247721 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-17ce90d3-fabb-4cc1-bed6-e5648e0ef5ce None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:00:55.249483 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-17ce90d3-fabb-4cc1-bed6-e5648e0ef5ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:55.275640 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:55.335106 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 235/474] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:00:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 03:00:55.336019 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ce90d3-fabb-4cc1-bed6-e5648e0ef5ce None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:55.352003 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6987ff53-e423-401e-8bf7-a28ee4087c78 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:55.425002 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606ef087-9a5e-463f-879c-6f1d07837af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:55.425933 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606ef087-9a5e-463f-879c-6f1d07837af3 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:55.427202 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606ef087-9a5e-463f-879c-6f1d07837af3 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:55.427698 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606ef087-9a5e-463f-879c-6f1d07837af3 None admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:55.429297 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606ef087-9a5e-463f-879c-6f1d07837af3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:55.429922 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606ef087-9a5e-463f-879c-6f1d07837af3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:55.432428 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606ef087-9a5e-463f-879c-6f1d07837af3 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:55.460286 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 240/475] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Aug 25 03:00:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:00:55.463909 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606ef087-9a5e-463f-879c-6f1d07837af3 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:55.495708 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ce90d3-fabb-4cc1-bed6-e5648e0ef5ce None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:55.536636 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a66c5ef-16a2-4b5a-9afa-64c0b14949b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:55.537618 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a66c5ef-16a2-4b5a-9afa-64c0b14949b2 None admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:55.538099 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a66c5ef-16a2-4b5a-9afa-64c0b14949b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:55.538600 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a66c5ef-16a2-4b5a-9afa-64c0b14949b2 None admin] PATH_INFO: `/v3/users` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:55.539999 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a66c5ef-16a2-4b5a-9afa-64c0b14949b2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:55.541757 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a66c5ef-16a2-4b5a-9afa-64c0b14949b2 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:55.583958 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:55.624982 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606ef087-9a5e-463f-879c-6f1d07837af3 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:55.663135 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 236/476] 10.222.0.58 () {60 vars in 1132 bytes} [Thu Aug 25 03:00:55 2022] POST /identity/v3/users => generated 249 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:00:55.669622 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a66c5ef-16a2-4b5a-9afa-64c0b14949b2 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:55.678388 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f88f636-407f-4c2d-917b-394b0a651179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:55.678388 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f88f636-407f-4c2d-917b-394b0a651179 None admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:55.678388 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f88f636-407f-4c2d-917b-394b0a651179 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:55.678388 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f88f636-407f-4c2d-917b-394b0a651179 None admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:55.678762 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f88f636-407f-4c2d-917b-394b0a651179 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:55.678762 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f88f636-407f-4c2d-917b-394b0a651179 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:55.712340 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:55.756186 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 241/477] 10.222.0.58 () {60 vars in 1132 bytes} [Thu Aug 25 03:00:55 2022] POST /identity/v3/users => generated 248 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:00:55.838952 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a66c5ef-16a2-4b5a-9afa-64c0b14949b2 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:55.881413 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f4b309-3b88-4d0d-9139-6e1027c47fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:55.882039 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f4b309-3b88-4d0d-9139-6e1027c47fcf None admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:55.882345 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f4b309-3b88-4d0d-9139-6e1027c47fcf None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:55.882624 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f4b309-3b88-4d0d-9139-6e1027c47fcf None admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:55.883304 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f4b309-3b88-4d0d-9139-6e1027c47fcf None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:55.884687 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f4b309-3b88-4d0d-9139-6e1027c47fcf None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:55.924151 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.sql.core [None req-f4f4b309-3b88-4d0d-9139-6e1027c47fcf None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 25 03:00:55.924151 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 25 03:00:55.924151 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [parameters: {'id': 'e44d229706fe46a8988b4b077f1fc6ed', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 25 03:00:55.924151 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81995) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 25 03:00:55.945664 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-f4f4b309-3b88-4d0d-9139-6e1027c47fcf None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 25 03:00:55.945664 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:00:55.945664 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 25 03:00:55.945664 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 25 03:00:55.945664 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 25 03:00:55.945664 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 25 03:00:55.945664 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 25 03:00:55.945664 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application result = self._query(query) Aug 25 03:00:55.945664 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 25 03:00:55.945664 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application conn.query(q) Aug 25 03:00:55.945664 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 25 03:00:55.945664 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 25 03:00:55.945664 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 25 03:00:55.945664 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application result.read() Aug 25 03:00:55.945664 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 25 03:00:55.946591 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 25 03:00:55.946591 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 25 03:00:55.946591 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 25 03:00:55.946591 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 25 03:00:55.946591 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 25 03:00:55.946591 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 25 03:00:55.946591 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 25 03:00:55.946591 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 25 03:00:55.946591 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:00:55.946591 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 25 03:00:55.946591 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:00:55.946591 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:00:55.946591 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 25 03:00:55.946591 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 25 03:00:55.946591 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 25 03:00:55.946591 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return ref.to_dict() Aug 25 03:00:55.946591 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 25 03:00:55.946591 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application next(self.gen) Aug 25 03:00:55.947503 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 25 03:00:55.947503 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application yield resource Aug 25 03:00:55.947503 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 25 03:00:55.947503 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application next(self.gen) Aug 25 03:00:55.947503 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 25 03:00:55.947503 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application self.session.rollback() Aug 25 03:00:55.947503 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 25 03:00:55.947503 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application self.force_reraise() Aug 25 03:00:55.947503 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 25 03:00:55.947503 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise self.value Aug 25 03:00:55.947503 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 25 03:00:55.947503 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 25 03:00:55.947503 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 25 03:00:55.947503 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application session.commit() Aug 25 03:00:55.947503 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 25 03:00:55.947503 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 25 03:00:55.947503 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 25 03:00:55.948352 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application self._prepare_impl() Aug 25 03:00:55.948352 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 25 03:00:55.948352 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application self.session.flush() Aug 25 03:00:55.948352 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 25 03:00:55.948352 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application self._flush(objects) Aug 25 03:00:55.948352 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 25 03:00:55.948352 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 25 03:00:55.948352 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 25 03:00:55.948352 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application compat.raise_( Aug 25 03:00:55.948352 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 25 03:00:55.948352 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception Aug 25 03:00:55.948352 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 25 03:00:55.948352 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application flush_context.execute() Aug 25 03:00:55.948352 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 25 03:00:55.948352 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rec.execute(self) Aug 25 03:00:55.948352 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 25 03:00:55.948352 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 25 03:00:55.949254 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 25 03:00:55.949254 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 25 03:00:55.949254 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 25 03:00:55.949254 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 25 03:00:55.949254 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 25 03:00:55.949254 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 25 03:00:55.949254 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 25 03:00:55.949254 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 25 03:00:55.949254 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 25 03:00:55.949254 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 25 03:00:55.949254 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 25 03:00:55.949254 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 25 03:00:55.949254 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 25 03:00:55.949254 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 25 03:00:55.949254 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 25 03:00:55.949254 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception Aug 25 03:00:55.950061 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 25 03:00:55.950061 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 25 03:00:55.950061 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 25 03:00:55.950061 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 25 03:00:55.950061 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 25 03:00:55.950061 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application result = self._query(query) Aug 25 03:00:55.950061 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 25 03:00:55.950061 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application conn.query(q) Aug 25 03:00:55.950061 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 25 03:00:55.950061 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 25 03:00:55.950061 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 25 03:00:55.950061 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application result.read() Aug 25 03:00:55.950061 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 25 03:00:55.950061 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 25 03:00:55.950061 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 25 03:00:55.950061 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 25 03:00:55.953367 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 25 03:00:55.953367 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 25 03:00:55.953367 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 25 03:00:55.953367 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 25 03:00:55.953367 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 25 03:00:55.953367 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 25 03:00:55.953367 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [parameters: {'id': 'e44d229706fe46a8988b4b077f1fc6ed', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 25 03:00:55.953367 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 25 03:00:55.953367 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:00:55.953367 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 25 03:00:55.953367 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:00:55.953367 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:00:55.953367 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:00:55.953367 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:00:55.953367 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:00:55.953367 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:00:55.955474 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:00:55.955474 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:00:55.955474 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:00:55.955474 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:00:55.955474 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:00:55.955474 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:00:55.955474 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 25 03:00:55.955474 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 25 03:00:55.955474 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:00:55.955474 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:00:55.955474 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 25 03:00:55.955474 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 25 03:00:55.955474 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 25 03:00:55.955474 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 25 03:00:55.955474 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 25 03:00:55.955474 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:00:55.957594 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 237/478] 10.222.0.58 () {60 vars in 1131 bytes} [Thu Aug 25 03:00:55 2022] POST /identity/v3/roles => generated 144 bytes in 283 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:00:55.957743 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f88f636-407f-4c2d-917b-394b0a651179 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:56.081532 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f88f636-407f-4c2d-917b-394b0a651179 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:56.099250 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:56.118557 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2519f991-e888-4a7d-8631-579c641523d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:56.119273 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2519f991-e888-4a7d-8631-579c641523d7 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:56.119602 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2519f991-e888-4a7d-8631-579c641523d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:56.119879 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2519f991-e888-4a7d-8631-579c641523d7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:56.132193 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2519f991-e888-4a7d-8631-579c641523d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:00:56.137124 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-2519f991-e888-4a7d-8631-579c641523d7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 25 03:00:56.137124 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:00:56.137124 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:00:56.137124 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:00:56.137124 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:00:56.137124 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:00:56.137124 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:00:56.137124 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:00:56.137124 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:00:56.137124 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:00:56.137124 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:00:56.137124 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:00:56.137124 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:00:56.137124 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:00:56.137124 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:00:56.137124 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:00:56.140366 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:00:56.140366 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:00:56.140366 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:00:56.140366 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:00:56.140366 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:00:56.140366 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:00:56.140366 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:00:56.140366 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:00:56.140366 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:00:56.140366 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:00:56.140366 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:00:56.140366 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:00:56.140366 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:00:56.140366 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:00:56.140366 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:00:56.140366 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:00:56.142375 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:00:56.142375 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:00:56.142375 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:00:56.142375 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:00:56.142375 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:00:56.142375 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:00:56.142375 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:00:56.142375 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:00:56.142375 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:00:56.142375 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:00:56.142375 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 25 03:00:56.142375 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:00:56.143961 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 242/479] 10.222.0.58 () {56 vars in 1114 bytes} [Thu Aug 25 03:00:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:00:56.150447 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f4b309-3b88-4d0d-9139-6e1027c47fcf None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:56.234746 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:56.258549 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f4b309-3b88-4d0d-9139-6e1027c47fcf None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:56.296518 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eb0618-c7f6-42ac-83a8-18da89d5a598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:56.297139 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eb0618-c7f6-42ac-83a8-18da89d5a598 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:56.297423 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eb0618-c7f6-42ac-83a8-18da89d5a598 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:56.297701 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eb0618-c7f6-42ac-83a8-18da89d5a598 None admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:56.298406 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00eb0618-c7f6-42ac-83a8-18da89d5a598 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:56.298719 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00eb0618-c7f6-42ac-83a8-18da89d5a598 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:56.300136 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00eb0618-c7f6-42ac-83a8-18da89d5a598 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:56.308891 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 238/480] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Aug 25 03:00:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:57.335413 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08877a9-3b1b-4e5a-8d9d-7329711eb660 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:57.336423 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08877a9-3b1b-4e5a-8d9d-7329711eb660 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:57.336934 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08877a9-3b1b-4e5a-8d9d-7329711eb660 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:57.338753 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 243/481] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:00:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:00:57.343651 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f186eeb-b303-4a4a-a3e9-114fc8c7e1fc None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:57.344440 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f186eeb-b303-4a4a-a3e9-114fc8c7e1fc None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:57.344918 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f186eeb-b303-4a4a-a3e9-114fc8c7e1fc None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:57.346470 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 239/482] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:00:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:00:57.348838 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414452ff-3712-4d52-af35-06ac8f883036 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:57.349325 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414452ff-3712-4d52-af35-06ac8f883036 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:57.349793 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414452ff-3712-4d52-af35-06ac8f883036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:57.351381 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1130a5-27cd-4b99-8827-706373556786 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:57.351669 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1130a5-27cd-4b99-8827-706373556786 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:57.351948 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1130a5-27cd-4b99-8827-706373556786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:57.377285 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-414452ff-3712-4d52-af35-06ac8f883036 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:00:57.378519 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-414452ff-3712-4d52-af35-06ac8f883036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:57.384643 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-2c1130a5-27cd-4b99-8827-706373556786 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:00:57.385816 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-2c1130a5-27cd-4b99-8827-706373556786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:57.505156 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 244/483] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:00:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 03:00:57.508872 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 240/484] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:00:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 03:00:57.788286 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414452ff-3712-4d52-af35-06ac8f883036 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:57.915388 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414452ff-3712-4d52-af35-06ac8f883036 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:57.928111 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:57.934901 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1130a5-27cd-4b99-8827-706373556786 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:57.980361 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920c56d2-14e5-4454-b250-16b90a990c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:57.981278 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920c56d2-14e5-4454-b250-16b90a990c13 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:57.982406 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920c56d2-14e5-4454-b250-16b90a990c13 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:57.983944 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920c56d2-14e5-4454-b250-16b90a990c13 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:58.003575 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920c56d2-14e5-4454-b250-16b90a990c13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:00:58.008386 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-920c56d2-14e5-4454-b250-16b90a990c13 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 25 03:00:58.008386 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:00:58.008386 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:00:58.008386 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:00:58.008386 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:00:58.008386 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:00:58.008386 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:00:58.008386 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:00:58.008386 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:00:58.008386 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:00:58.008386 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:00:58.008386 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:00:58.008386 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:00:58.008386 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:00:58.008386 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:00:58.008386 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:00:58.009441 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:00:58.009441 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:00:58.009441 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:00:58.009441 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:00:58.009441 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:00:58.009441 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:00:58.009441 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:00:58.009441 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:00:58.009441 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:00:58.009441 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:00:58.009441 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:00:58.009441 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:00:58.009441 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:00:58.009441 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:00:58.009441 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:00:58.009441 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:00:58.014602 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:00:58.014602 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:00:58.014602 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:00:58.014602 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:00:58.014602 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:00:58.014602 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:00:58.014602 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:00:58.014602 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:00:58.014602 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:00:58.014602 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:00:58.014602 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 25 03:00:58.014602 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:00:58.014602 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 245/485] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Aug 25 03:00:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:00:58.027633 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920c56d2-14e5-4454-b250-16b90a990c13 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:58.039556 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:58.065179 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1130a5-27cd-4b99-8827-706373556786 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:58.101939 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479b9d2a-45cb-4faf-bf7f-05c0fc40b235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:58.102895 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479b9d2a-45cb-4faf-bf7f-05c0fc40b235 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:58.103357 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479b9d2a-45cb-4faf-bf7f-05c0fc40b235 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:58.103838 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479b9d2a-45cb-4faf-bf7f-05c0fc40b235 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:58.115081 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479b9d2a-45cb-4faf-bf7f-05c0fc40b235 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:00:58.120641 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-479b9d2a-45cb-4faf-bf7f-05c0fc40b235 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 25 03:00:58.120641 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:00:58.120641 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:00:58.120641 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:00:58.120641 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:00:58.120641 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:00:58.120641 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:00:58.120641 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:00:58.120641 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:00:58.120641 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:00:58.120641 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:00:58.120641 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:00:58.120641 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:00:58.120641 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:00:58.120641 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:00:58.120641 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:00:58.121509 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:00:58.121509 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:00:58.121509 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:00:58.121509 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:00:58.121509 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:00:58.121509 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:00:58.121509 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:00:58.121509 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:00:58.121509 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:00:58.121509 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:00:58.121509 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:00:58.121509 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:00:58.121509 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:00:58.121509 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:00:58.121509 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:00:58.121509 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:00:58.122524 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:00:58.122524 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:00:58.122524 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:00:58.122524 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:00:58.122524 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:00:58.122524 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:00:58.122524 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:00:58.122524 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:00:58.122524 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:00:58.122524 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:00:58.122524 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 25 03:00:58.122524 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:00:58.125449 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 241/486] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Aug 25 03:00:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:00:58.130894 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479b9d2a-45cb-4faf-bf7f-05c0fc40b235 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:58.233385 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920c56d2-14e5-4454-b250-16b90a990c13 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:58.248286 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479b9d2a-45cb-4faf-bf7f-05c0fc40b235 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:58.268049 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afe4054-1d84-459b-ba23-dca90292206e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:58.268651 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afe4054-1d84-459b-ba23-dca90292206e None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:58.268930 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afe4054-1d84-459b-ba23-dca90292206e None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:58.269200 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afe4054-1d84-459b-ba23-dca90292206e None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:58.269852 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afe4054-1d84-459b-ba23-dca90292206e None admin] RBAC: Adding query filter params (name=service) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:58.270148 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afe4054-1d84-459b-ba23-dca90292206e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:58.271538 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afe4054-1d84-459b-ba23-dca90292206e None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:58.279694 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 246/487] 10.222.0.58 () {56 vars in 1103 bytes} [Thu Aug 25 03:00:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:00:58.280332 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:58.280725 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7474881b-e069-40e6-b47e-dcb972b91228 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:58.281006 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7474881b-e069-40e6-b47e-dcb972b91228 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:58.281632 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7474881b-e069-40e6-b47e-dcb972b91228 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:58.283004 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 247/488] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:00:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:00:58.287858 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66ac070-50d6-40bc-807c-e3e61061dfb3 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:58.288153 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66ac070-50d6-40bc-807c-e3e61061dfb3 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:58.288545 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66ac070-50d6-40bc-807c-e3e61061dfb3 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:58.289725 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 248/489] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:00:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:00:58.289945 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1116dd3c-9dab-48cc-a5e5-64b34529e14c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:58.294745 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66ac070-50d6-40bc-807c-e3e61061dfb3 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:58.294851 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1116dd3c-9dab-48cc-a5e5-64b34529e14c None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:58.294851 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1116dd3c-9dab-48cc-a5e5-64b34529e14c None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:58.294851 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1116dd3c-9dab-48cc-a5e5-64b34529e14c None admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:58.294851 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1116dd3c-9dab-48cc-a5e5-64b34529e14c None admin] RBAC: Adding query filter params (name=service) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:58.294851 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1116dd3c-9dab-48cc-a5e5-64b34529e14c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:58.294851 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1116dd3c-9dab-48cc-a5e5-64b34529e14c None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:58.302447 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 242/490] 10.222.0.58 () {56 vars in 1103 bytes} [Thu Aug 25 03:00:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:00:58.303812 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd79b83-2c10-4fc3-8d1a-20546b2d52f5 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:58.304229 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd79b83-2c10-4fc3-8d1a-20546b2d52f5 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:58.304675 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd79b83-2c10-4fc3-8d1a-20546b2d52f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:58.323359 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-2cd79b83-2c10-4fc3-8d1a-20546b2d52f5 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:00:58.324539 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-2cd79b83-2c10-4fc3-8d1a-20546b2d52f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:58.330838 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:58.393208 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 243/491] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:00:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 03:00:58.403774 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d4ced1-5206-4850-a9e8-35fa2288da3b None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:58.404075 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d4ced1-5206-4850-a9e8-35fa2288da3b None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:58.404358 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d4ced1-5206-4850-a9e8-35fa2288da3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:58.410123 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66ac070-50d6-40bc-807c-e3e61061dfb3 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:58.423624 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-09d4ced1-5206-4850-a9e8-35fa2288da3b None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:00:58.424861 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-09d4ced1-5206-4850-a9e8-35fa2288da3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:58.447293 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d53ab5-6bc3-4fa3-b6bd-a9babf245de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:58.448320 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d53ab5-6bc3-4fa3-b6bd-a9babf245de6 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:58.448797 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d53ab5-6bc3-4fa3-b6bd-a9babf245de6 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:58.449314 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d53ab5-6bc3-4fa3-b6bd-a9babf245de6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:58.460349 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d53ab5-6bc3-4fa3-b6bd-a9babf245de6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:58.462054 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d53ab5-6bc3-4fa3-b6bd-a9babf245de6 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:58.477354 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-28d53ab5-6bc3-4fa3-b6bd-a9babf245de6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:00:58.477354 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:00:58.477354 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:00:58.477354 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:00:58.477354 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:00:58.477354 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:00:58.477354 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:00:58.477354 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:00:58.477354 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:00:58.477354 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:00:58.477354 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:00:58.477354 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:00:58.477354 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 03:00:58.477354 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 03:00:58.477354 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 03:00:58.477354 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 03:00:58.478553 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:00:58.478553 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:00:58.478553 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:00:58.478553 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:00:58.478553 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:00:58.478553 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:00:58.478553 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:00:58.478553 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:00:58.478553 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:00:58.478553 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:00:58.478553 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:00:58.478553 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:00:58.478553 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:00:58.478553 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:00:58.478553 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:00:58.478553 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:00:58.478553 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 03:00:58.479397 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 03:00:58.479397 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:00:58.479397 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:00:58.486265 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 249/492] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:00:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:00:58.487380 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d53ab5-6bc3-4fa3-b6bd-a9babf245de6 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:58.506564 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 244/493] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:00:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 03:00:58.507629 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d4ced1-5206-4850-a9e8-35fa2288da3b None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:58.572319 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:58.592134 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d53ab5-6bc3-4fa3-b6bd-a9babf245de6 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:58.606661 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:58.627841 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3814a987-73b5-438e-a0e3-4b7af52194af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:58.628849 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3814a987-73b5-438e-a0e3-4b7af52194af None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:58.629342 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3814a987-73b5-438e-a0e3-4b7af52194af None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:58.629804 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3814a987-73b5-438e-a0e3-4b7af52194af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:58.645071 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3814a987-73b5-438e-a0e3-4b7af52194af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:58.647040 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3814a987-73b5-438e-a0e3-4b7af52194af None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:58.678991 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-3814a987-73b5-438e-a0e3-4b7af52194af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:00:58.678991 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:00:58.678991 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:00:58.678991 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:00:58.678991 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:00:58.678991 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:00:58.678991 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:00:58.678991 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:00:58.678991 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:00:58.678991 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:00:58.678991 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:00:58.678991 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:00:58.678991 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 03:00:58.678991 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 03:00:58.678991 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 03:00:58.678991 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 03:00:58.679921 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:00:58.679921 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:00:58.679921 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:00:58.679921 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:00:58.679921 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:00:58.679921 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:00:58.679921 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:00:58.679921 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:00:58.679921 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:00:58.679921 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:00:58.679921 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:00:58.679921 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:00:58.679921 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:00:58.679921 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:00:58.679921 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:00:58.679921 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:00:58.679921 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 03:00:58.680742 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 03:00:58.680742 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:00:58.680742 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:00:58.680742 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 250/494] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:00:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:00:58.681418 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3814a987-73b5-438e-a0e3-4b7af52194af None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:58.687486 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d4ced1-5206-4850-a9e8-35fa2288da3b None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:58.725524 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790f6e54-f361-4c69-893b-270eae327a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:58.726473 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790f6e54-f361-4c69-893b-270eae327a0c None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:58.726919 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790f6e54-f361-4c69-893b-270eae327a0c None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:58.727354 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790f6e54-f361-4c69-893b-270eae327a0c None admin] PATH_INFO: `/v3/domains` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:58.729307 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790f6e54-f361-4c69-893b-270eae327a0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:58.729805 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790f6e54-f361-4c69-893b-270eae327a0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:58.731419 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790f6e54-f361-4c69-893b-270eae327a0c None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:58.746975 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 245/495] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Aug 25 03:00:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:58.753587 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790f6e54-f361-4c69-893b-270eae327a0c None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:58.803863 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3814a987-73b5-438e-a0e3-4b7af52194af None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:58.845069 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9cd644-ab5e-4d27-8955-47b5c3aa6363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:58.846148 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9cd644-ab5e-4d27-8955-47b5c3aa6363 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:58.846604 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9cd644-ab5e-4d27-8955-47b5c3aa6363 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:58.847015 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9cd644-ab5e-4d27-8955-47b5c3aa6363 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:58.848954 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:58.858648 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9cd644-ab5e-4d27-8955-47b5c3aa6363 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:00:58.862383 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790f6e54-f361-4c69-893b-270eae327a0c None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:58.864351 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-7c9cd644-ab5e-4d27-8955-47b5c3aa6363 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 25 03:00:58.864351 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:00:58.864351 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:00:58.864351 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:00:58.864351 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:00:58.864351 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:00:58.864351 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:00:58.864351 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:00:58.864351 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:00:58.864351 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:00:58.864351 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:00:58.864351 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:00:58.864351 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:00:58.864351 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:00:58.864351 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:00:58.864351 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:00:58.865219 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:00:58.865219 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:00:58.865219 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:00:58.865219 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:00:58.865219 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:00:58.865219 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:00:58.865219 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:00:58.865219 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:00:58.865219 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:00:58.865219 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:00:58.865219 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:00:58.865219 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:00:58.865219 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:00:58.865219 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:00:58.865219 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:00:58.865219 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:00:58.866002 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:00:58.866002 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:00:58.866002 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:00:58.866002 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:00:58.866002 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:00:58.866002 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:00:58.866002 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:00:58.866002 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:00:58.866002 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:00:58.866002 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:00:58.866002 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 25 03:00:58.866002 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:00:58.866002 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 251/496] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Aug 25 03:00:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:00:58.867893 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9cd644-ab5e-4d27-8955-47b5c3aa6363 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:58.880998 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:58.902017 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992a3a7a-27ad-4322-aad3-098827a85c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:58.902666 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992a3a7a-27ad-4322-aad3-098827a85c5c None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:58.903096 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992a3a7a-27ad-4322-aad3-098827a85c5c None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:58.903674 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992a3a7a-27ad-4322-aad3-098827a85c5c None admin] PATH_INFO: `/v3/domains` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:58.904340 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992a3a7a-27ad-4322-aad3-098827a85c5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:58.904653 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992a3a7a-27ad-4322-aad3-098827a85c5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:58.906031 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992a3a7a-27ad-4322-aad3-098827a85c5c None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:58.917906 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 246/497] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Aug 25 03:00:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:58.919195 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0584a9-1b87-4c93-b919-21e2364aa9ac None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:58.919536 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0584a9-1b87-4c93-b919-21e2364aa9ac None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:58.919819 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0584a9-1b87-4c93-b919-21e2364aa9ac None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:58.921011 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 247/498] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:00:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:00:58.922057 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0584a9-1b87-4c93-b919-21e2364aa9ac None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:58.967553 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9cd644-ab5e-4d27-8955-47b5c3aa6363 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:59.001851 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65483e6f-a8c2-4b0f-9929-bce46c453c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:59.002609 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65483e6f-a8c2-4b0f-9929-bce46c453c7c None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:59.002891 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65483e6f-a8c2-4b0f-9929-bce46c453c7c None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:59.003163 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65483e6f-a8c2-4b0f-9929-bce46c453c7c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:59.010559 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65483e6f-a8c2-4b0f-9929-bce46c453c7c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:59.012001 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65483e6f-a8c2-4b0f-9929-bce46c453c7c None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:59.021584 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0584a9-1b87-4c93-b919-21e2364aa9ac None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:59.024835 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-65483e6f-a8c2-4b0f-9929-bce46c453c7c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 25 03:00:59.024835 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:00:59.024835 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:00:59.024835 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:00:59.024835 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:00:59.024835 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:00:59.024835 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:00:59.024835 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:00:59.024835 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:00:59.024835 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:00:59.024835 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:00:59.024835 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:00:59.024835 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 25 03:00:59.024835 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 25 03:00:59.024835 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 25 03:00:59.024835 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 25 03:00:59.025717 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:00:59.025717 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:00:59.025717 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 25 03:00:59.025717 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 03:00:59.025717 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 25 03:00:59.025717 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 03:00:59.025717 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:00:59.025717 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:00:59.025717 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:00:59.025717 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:00:59.025717 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:00:59.025717 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:00:59.025717 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:00:59.025717 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:00:59.025717 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:00:59.025717 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:00:59.025717 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:00:59.025717 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:00:59.026616 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:00:59.026616 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:00:59.026616 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 25 03:00:59.026616 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 25 03:00:59.026616 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 25 03:00:59.026616 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 25 03:00:59.026616 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 25 03:00:59.026616 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 25 03:00:59.026616 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 25 03:00:59.026616 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:00:59.026616 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 252/499] 10.222.0.58 () {56 vars in 1093 bytes} [Thu Aug 25 03:00:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:00:59.027166 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65483e6f-a8c2-4b0f-9929-bce46c453c7c None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:59.057684 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302bba18-2c0a-4306-af15-8f5159d43066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:59.058329 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302bba18-2c0a-4306-af15-8f5159d43066 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:59.058613 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302bba18-2c0a-4306-af15-8f5159d43066 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:59.058885 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302bba18-2c0a-4306-af15-8f5159d43066 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:59.069371 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302bba18-2c0a-4306-af15-8f5159d43066 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:00:59.074118 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-302bba18-2c0a-4306-af15-8f5159d43066 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 25 03:00:59.074118 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:00:59.074118 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:00:59.074118 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:00:59.074118 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:00:59.074118 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:00:59.074118 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:00:59.074118 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:00:59.074118 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:00:59.074118 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:00:59.074118 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:00:59.074118 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:00:59.074118 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:00:59.074118 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:00:59.074118 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:00:59.074118 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:00:59.075181 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:00:59.075181 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:00:59.075181 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:00:59.075181 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:00:59.075181 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:00:59.075181 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:00:59.075181 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:00:59.075181 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:00:59.075181 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:00:59.075181 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:00:59.075181 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:00:59.075181 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:00:59.075181 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:00:59.075181 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:00:59.075181 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:00:59.075181 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:00:59.075985 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:00:59.075985 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:00:59.075985 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:00:59.075985 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:00:59.075985 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:00:59.075985 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:00:59.075985 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:00:59.075985 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:00:59.075985 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:00:59.075985 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:00:59.075985 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 25 03:00:59.075985 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:00:59.075985 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 248/500] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Aug 25 03:00:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:00:59.075985 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302bba18-2c0a-4306-af15-8f5159d43066 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:59.083582 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:59.106153 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:59.128457 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65483e6f-a8c2-4b0f-9929-bce46c453c7c None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:59.160998 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f944d7f9-866b-457d-ab5e-1465f35fdafd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:59.161607 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f944d7f9-866b-457d-ab5e-1465f35fdafd None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:59.161887 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f944d7f9-866b-457d-ab5e-1465f35fdafd None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:59.162160 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f944d7f9-866b-457d-ab5e-1465f35fdafd None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:59.162847 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f944d7f9-866b-457d-ab5e-1465f35fdafd None admin] RBAC: Adding query filter params (name=service) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:59.163151 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f944d7f9-866b-457d-ab5e-1465f35fdafd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:59.164522 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f944d7f9-866b-457d-ab5e-1465f35fdafd None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:59.172952 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 253/501] 10.222.0.58 () {56 vars in 1103 bytes} [Thu Aug 25 03:00:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:59.173605 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302bba18-2c0a-4306-af15-8f5159d43066 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:59.174226 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b045cbe6-9931-411d-82e8-f986ab06b1c6 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:59.174602 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b045cbe6-9931-411d-82e8-f986ab06b1c6 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:59.174944 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b045cbe6-9931-411d-82e8-f986ab06b1c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:59.192564 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-b045cbe6-9931-411d-82e8-f986ab06b1c6 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:00:59.193763 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-b045cbe6-9931-411d-82e8-f986ab06b1c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:59.210864 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3274be-94e6-4b47-bdd5-5b22bfe3c088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:59.211494 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3274be-94e6-4b47-bdd5-5b22bfe3c088 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:59.211827 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3274be-94e6-4b47-bdd5-5b22bfe3c088 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:59.212100 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3274be-94e6-4b47-bdd5-5b22bfe3c088 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:59.219264 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3274be-94e6-4b47-bdd5-5b22bfe3c088 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:59.220707 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3274be-94e6-4b47-bdd5-5b22bfe3c088 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:59.232993 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-ee3274be-94e6-4b47-bdd5-5b22bfe3c088 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 25 03:00:59.232993 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:00:59.232993 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:00:59.232993 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:00:59.232993 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:00:59.232993 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:00:59.232993 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:00:59.232993 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:00:59.232993 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:00:59.232993 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:00:59.232993 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:00:59.232993 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:00:59.232993 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 25 03:00:59.232993 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 25 03:00:59.232993 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 25 03:00:59.232993 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 25 03:00:59.234612 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:00:59.234612 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:00:59.234612 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 25 03:00:59.234612 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 03:00:59.234612 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 25 03:00:59.234612 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 03:00:59.234612 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:00:59.234612 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:00:59.234612 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:00:59.234612 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:00:59.234612 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:00:59.234612 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:00:59.234612 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:00:59.234612 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:00:59.234612 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:00:59.234612 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:00:59.234612 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:00:59.234612 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:00:59.235510 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:00:59.235510 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:00:59.235510 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 25 03:00:59.235510 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 25 03:00:59.235510 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 25 03:00:59.235510 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 25 03:00:59.235510 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 25 03:00:59.235510 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 25 03:00:59.235510 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 25 03:00:59.235510 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:00:59.235510 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 249/502] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Aug 25 03:00:59 2022] GET /identity/v3/users/glance => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:00:59.235510 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3274be-94e6-4b47-bdd5-5b22bfe3c088 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:59.263645 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 254/503] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:00:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 03:00:59.264412 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b045cbe6-9931-411d-82e8-f986ab06b1c6 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:59.310980 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:59.337224 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:59.346363 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3274be-94e6-4b47-bdd5-5b22bfe3c088 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:59.371889 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b045cbe6-9931-411d-82e8-f986ab06b1c6 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:59.387279 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4175839e-697a-4a2b-b876-eb0680db21ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:59.387981 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4175839e-697a-4a2b-b876-eb0680db21ed None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:59.388379 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4175839e-697a-4a2b-b876-eb0680db21ed None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:59.388692 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4175839e-697a-4a2b-b876-eb0680db21ed None admin] PATH_INFO: `/v3/users` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:59.389578 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4175839e-697a-4a2b-b876-eb0680db21ed None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:59.389960 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4175839e-697a-4a2b-b876-eb0680db21ed None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:59.391518 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4175839e-697a-4a2b-b876-eb0680db21ed None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:59.408401 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b8844b-62b8-41ef-917e-4110d3e3357c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:59.409010 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b8844b-62b8-41ef-917e-4110d3e3357c None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:59.409288 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b8844b-62b8-41ef-917e-4110d3e3357c None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:59.409565 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b8844b-62b8-41ef-917e-4110d3e3357c None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:59.410233 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b8844b-62b8-41ef-917e-4110d3e3357c None admin] RBAC: Adding query filter params (name=service) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:59.410770 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b8844b-62b8-41ef-917e-4110d3e3357c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:59.411912 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b8844b-62b8-41ef-917e-4110d3e3357c None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:59.425069 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 255/504] 10.222.0.58 () {56 vars in 1103 bytes} [Thu Aug 25 03:00:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:59.425069 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b8844b-62b8-41ef-917e-4110d3e3357c None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:59.429035 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 250/505] 10.222.0.58 () {56 vars in 1133 bytes} [Thu Aug 25 03:00:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:59.434116 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4175839e-697a-4a2b-b876-eb0680db21ed None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:59.524356 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b8844b-62b8-41ef-917e-4110d3e3357c None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:59.533832 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:59.540426 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4175839e-697a-4a2b-b876-eb0680db21ed None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:59.558724 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5d8550-d9c6-4036-8fb9-a08f83926407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:59.559358 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5d8550-d9c6-4036-8fb9-a08f83926407 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:59.559638 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5d8550-d9c6-4036-8fb9-a08f83926407 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:59.559911 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5d8550-d9c6-4036-8fb9-a08f83926407 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:59.570782 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5d8550-d9c6-4036-8fb9-a08f83926407 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:59.572295 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5d8550-d9c6-4036-8fb9-a08f83926407 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:59.574836 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c089675-c2e9-4e77-a382-466473056ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:59.575437 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c089675-c2e9-4e77-a382-466473056ad8 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:59.575715 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c089675-c2e9-4e77-a382-466473056ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:59.575987 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c089675-c2e9-4e77-a382-466473056ad8 None admin] PATH_INFO: `/v3/users` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:59.576714 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c089675-c2e9-4e77-a382-466473056ad8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:00:59.577801 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c089675-c2e9-4e77-a382-466473056ad8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:59.579273 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c089675-c2e9-4e77-a382-466473056ad8 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:59.588622 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-fb5d8550-d9c6-4036-8fb9-a08f83926407 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:00:59.588622 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:00:59.588622 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:00:59.588622 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:00:59.588622 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:00:59.588622 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:00:59.588622 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:00:59.588622 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:00:59.588622 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:00:59.588622 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:00:59.588622 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:00:59.588622 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:00:59.588622 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 03:00:59.588622 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 03:00:59.588622 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 03:00:59.588622 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 03:00:59.589515 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:00:59.589515 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:00:59.589515 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:00:59.589515 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:00:59.589515 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:00:59.589515 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:00:59.589515 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:00:59.589515 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:00:59.589515 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:00:59.589515 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:00:59.589515 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:00:59.589515 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:00:59.589515 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:00:59.589515 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:00:59.589515 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:00:59.589515 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:00:59.589515 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 03:00:59.590410 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 03:00:59.590410 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:00:59.590410 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:00:59.590410 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 256/506] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:00:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:00:59.591009 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5d8550-d9c6-4036-8fb9-a08f83926407 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:59.602061 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 251/507] 10.222.0.58 () {56 vars in 1137 bytes} [Thu Aug 25 03:00:59 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:00:59.602061 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c089675-c2e9-4e77-a382-466473056ad8 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:59.609758 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:59.694371 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5d8550-d9c6-4036-8fb9-a08f83926407 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:59.697411 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c089675-c2e9-4e77-a382-466473056ad8 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:59.727349 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53247bb5-5ea3-4c23-aa55-eea98a551fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:59.727992 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53247bb5-5ea3-4c23-aa55-eea98a551fa5 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:59.728251 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53247bb5-5ea3-4c23-aa55-eea98a551fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:59.728522 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53247bb5-5ea3-4c23-aa55-eea98a551fa5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:59.731626 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6961286b-a6b7-4018-a486-abdd5e97e866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:59.732258 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6961286b-a6b7-4018-a486-abdd5e97e866 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:59.732543 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6961286b-a6b7-4018-a486-abdd5e97e866 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:59.732816 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6961286b-a6b7-4018-a486-abdd5e97e866 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:59.738714 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53247bb5-5ea3-4c23-aa55-eea98a551fa5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:59.740154 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53247bb5-5ea3-4c23-aa55-eea98a551fa5 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:59.754613 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-53247bb5-5ea3-4c23-aa55-eea98a551fa5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:00:59.754613 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:00:59.754613 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:00:59.754613 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:00:59.754613 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:00:59.754613 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:00:59.754613 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:00:59.754613 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:00:59.754613 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:00:59.754613 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:00:59.754613 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:00:59.754613 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:00:59.754613 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 03:00:59.754613 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 03:00:59.754613 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 03:00:59.754613 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 03:00:59.755575 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:00:59.755575 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:00:59.755575 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:00:59.755575 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:00:59.755575 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:00:59.755575 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:00:59.755575 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:00:59.755575 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:00:59.755575 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:00:59.755575 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:00:59.755575 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:00:59.755575 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:00:59.755575 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:00:59.755575 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:00:59.755575 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:00:59.755575 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:00:59.755575 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 03:00:59.757125 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 03:00:59.757125 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:00:59.757125 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:00:59.757125 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 257/508] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:00:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:00:59.758228 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53247bb5-5ea3-4c23-aa55-eea98a551fa5 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:59.773102 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:59.801941 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6961286b-a6b7-4018-a486-abdd5e97e866 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:59.803380 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6961286b-a6b7-4018-a486-abdd5e97e866 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:59.813556 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:59.855328 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53247bb5-5ea3-4c23-aa55-eea98a551fa5 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:59.888895 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5a6223-7c0c-4596-80df-189db1856e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:59.889583 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5a6223-7c0c-4596-80df-189db1856e4f None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:59.889864 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5a6223-7c0c-4596-80df-189db1856e4f None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:59.890140 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5a6223-7c0c-4596-80df-189db1856e4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:59.899899 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5a6223-7c0c-4596-80df-189db1856e4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:59.901208 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 252/509] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Aug 25 03:00:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 25 03:00:59.901350 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5a6223-7c0c-4596-80df-189db1856e4f None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:59.902033 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6961286b-a6b7-4018-a486-abdd5e97e866 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:59.918848 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-fc5a6223-7c0c-4596-80df-189db1856e4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:00:59.918848 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:00:59.918848 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:00:59.918848 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:00:59.918848 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:00:59.918848 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:00:59.918848 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:00:59.918848 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:00:59.918848 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:00:59.918848 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:00:59.918848 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:00:59.918848 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:00:59.918848 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 03:00:59.918848 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 03:00:59.918848 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 03:00:59.918848 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 03:00:59.919911 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:00:59.919911 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:00:59.919911 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:00:59.919911 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:00:59.919911 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:00:59.919911 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:00:59.919911 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:00:59.919911 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:00:59.919911 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:00:59.919911 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:00:59.919911 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:00:59.919911 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:00:59.919911 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:00:59.919911 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:00:59.919911 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:00:59.919911 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:00:59.919911 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 03:00:59.921281 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 03:00:59.921281 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:00:59.921281 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:00:59.921281 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 258/510] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:00:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:00:59.921281 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5a6223-7c0c-4596-80df-189db1856e4f None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:59.984376 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:00.002434 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6961286b-a6b7-4018-a486-abdd5e97e866 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:00.010856 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:00.033885 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4977b6f-01b5-43c3-bfd2-569053eef92e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:00.034518 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4977b6f-01b5-43c3-bfd2-569053eef92e None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:00.034797 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4977b6f-01b5-43c3-bfd2-569053eef92e None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:00.035069 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4977b6f-01b5-43c3-bfd2-569053eef92e None admin] PATH_INFO: `/v3/domains` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:00.035697 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4977b6f-01b5-43c3-bfd2-569053eef92e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:00.035995 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4977b6f-01b5-43c3-bfd2-569053eef92e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:00.037435 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4977b6f-01b5-43c3-bfd2-569053eef92e None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:00.049486 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 253/511] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Aug 25 03:00:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:00.050303 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4977b6f-01b5-43c3-bfd2-569053eef92e None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:00.069236 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5a6223-7c0c-4596-80df-189db1856e4f None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:00.101559 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d986f3a3-eb18-4b76-9f36-60957e3ea740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:00.102163 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d986f3a3-eb18-4b76-9f36-60957e3ea740 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:00.102479 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d986f3a3-eb18-4b76-9f36-60957e3ea740 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:00.102754 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d986f3a3-eb18-4b76-9f36-60957e3ea740 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:00.149517 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4977b6f-01b5-43c3-bfd2-569053eef92e None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:00.169408 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d986f3a3-eb18-4b76-9f36-60957e3ea740 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:00.171508 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d986f3a3-eb18-4b76-9f36-60957e3ea740 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:00.182114 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbb30e8-b18f-4e8b-9d40-ab2614174007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:00.182740 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbb30e8-b18f-4e8b-9d40-ab2614174007 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:00.183021 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbb30e8-b18f-4e8b-9d40-ab2614174007 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:00.183295 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbb30e8-b18f-4e8b-9d40-ab2614174007 None admin] PATH_INFO: `/v3/domains` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:00.183928 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbb30e8-b18f-4e8b-9d40-ab2614174007 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:00.184230 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbb30e8-b18f-4e8b-9d40-ab2614174007 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:00.185583 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbb30e8-b18f-4e8b-9d40-ab2614174007 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:00.196884 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 254/512] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Aug 25 03:01:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:00.197662 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbb30e8-b18f-4e8b-9d40-ab2614174007 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:00.226004 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:00.230533 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:00.267885 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 259/513] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Aug 25 03:00:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 25 03:01:00.268900 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d986f3a3-eb18-4b76-9f36-60957e3ea740 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:00.298131 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbb30e8-b18f-4e8b-9d40-ab2614174007 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:00.330972 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbe6eb0-a012-4805-bc3a-010fcf58de8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:00.331615 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbe6eb0-a012-4805-bc3a-010fcf58de8f None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:00.331891 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbe6eb0-a012-4805-bc3a-010fcf58de8f None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:00.332503 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbe6eb0-a012-4805-bc3a-010fcf58de8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:00.341702 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbe6eb0-a012-4805-bc3a-010fcf58de8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:00.343734 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbe6eb0-a012-4805-bc3a-010fcf58de8f None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:00.357498 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-1bbe6eb0-a012-4805-bc3a-010fcf58de8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:00.357498 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:00.357498 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:00.357498 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:00.357498 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:00.357498 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:00.357498 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:00.357498 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:00.357498 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:00.357498 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:00.357498 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:00.357498 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:00.357498 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 03:01:00.357498 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 03:01:00.357498 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 03:01:00.357498 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 03:01:00.359096 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:00.359096 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:00.359096 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:00.359096 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:00.359096 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:00.359096 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:00.359096 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:00.359096 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:00.359096 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:00.359096 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:00.359096 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:00.359096 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:00.359096 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:00.359096 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:00.359096 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:00.359096 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:00.359096 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 03:01:00.359927 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 03:01:00.359927 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:00.359927 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:00.359927 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 255/514] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:01:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:00.360145 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbe6eb0-a012-4805-bc3a-010fcf58de8f None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:00.366504 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d986f3a3-eb18-4b76-9f36-60957e3ea740 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:00.399117 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6439b942-23e4-4f77-bf38-88bcee9066fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:00.399718 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6439b942-23e4-4f77-bf38-88bcee9066fb None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:00.400005 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6439b942-23e4-4f77-bf38-88bcee9066fb None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:00.400278 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6439b942-23e4-4f77-bf38-88bcee9066fb None admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:00.400913 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6439b942-23e4-4f77-bf38-88bcee9066fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:00.401217 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6439b942-23e4-4f77-bf38-88bcee9066fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:00.402626 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6439b942-23e4-4f77-bf38-88bcee9066fb None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:00.414110 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 260/515] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Aug 25 03:01:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:00.415539 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6439b942-23e4-4f77-bf38-88bcee9066fb None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:00.440400 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:00.444807 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:00.462652 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbe6eb0-a012-4805-bc3a-010fcf58de8f None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:00.496058 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43163eb-7036-4c1c-97da-30a30201d057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:00.496678 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43163eb-7036-4c1c-97da-30a30201d057 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:00.496956 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43163eb-7036-4c1c-97da-30a30201d057 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:00.497225 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43163eb-7036-4c1c-97da-30a30201d057 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:00.504176 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43163eb-7036-4c1c-97da-30a30201d057 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:00.506261 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43163eb-7036-4c1c-97da-30a30201d057 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:00.513125 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6439b942-23e4-4f77-bf38-88bcee9066fb None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:00.518229 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-b43163eb-7036-4c1c-97da-30a30201d057 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 25 03:01:00.518229 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:00.518229 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:00.518229 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:00.518229 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:00.518229 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:00.518229 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:00.518229 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:00.518229 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:00.518229 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:00.518229 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:00.518229 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:00.518229 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 25 03:01:00.518229 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 25 03:01:00.518229 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 25 03:01:00.518229 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 25 03:01:00.519343 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:00.519343 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:00.519343 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 25 03:01:00.519343 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 03:01:00.519343 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 25 03:01:00.519343 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 03:01:00.519343 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:00.519343 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:00.519343 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:00.519343 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:00.519343 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:00.519343 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:00.519343 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:00.519343 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:00.519343 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:00.519343 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:00.519343 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:00.519343 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:00.520604 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:00.520604 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:00.520604 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 25 03:01:00.520604 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 25 03:01:00.520604 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 25 03:01:00.520604 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 25 03:01:00.520604 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 25 03:01:00.520604 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 25 03:01:00.520604 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 25 03:01:00.520604 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:00.520604 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 256/516] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Aug 25 03:01:00 2022] GET /identity/v3/users/cinder => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:00.520604 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43163eb-7036-4c1c-97da-30a30201d057 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:00.548312 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd1eb7e-080f-47e8-9cec-7e6fe0ebe4ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:00.548939 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd1eb7e-080f-47e8-9cec-7e6fe0ebe4ec None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:00.549243 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd1eb7e-080f-47e8-9cec-7e6fe0ebe4ec None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:00.549514 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd1eb7e-080f-47e8-9cec-7e6fe0ebe4ec None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:00.556870 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd1eb7e-080f-47e8-9cec-7e6fe0ebe4ec None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:00.558324 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd1eb7e-080f-47e8-9cec-7e6fe0ebe4ec None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:00.569348 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-8dd1eb7e-080f-47e8-9cec-7e6fe0ebe4ec None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 25 03:01:00.569348 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:00.569348 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:00.569348 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:00.569348 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:00.569348 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:00.569348 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:00.569348 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:00.569348 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:00.569348 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:00.569348 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:00.569348 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:00.569348 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 25 03:01:00.569348 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 25 03:01:00.569348 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 25 03:01:00.569348 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 25 03:01:00.570468 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:00.570468 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:00.570468 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 25 03:01:00.570468 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 03:01:00.570468 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 25 03:01:00.570468 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 03:01:00.570468 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:00.570468 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:00.570468 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:00.570468 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:00.570468 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:00.570468 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:00.570468 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:00.570468 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:00.570468 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:00.570468 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:00.570468 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:00.570468 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:00.571823 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:00.571823 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:00.571823 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 25 03:01:00.571823 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 25 03:01:00.571823 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 25 03:01:00.571823 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 25 03:01:00.571823 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 25 03:01:00.571823 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 25 03:01:00.571823 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 25 03:01:00.571823 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:00.571823 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 261/517] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Aug 25 03:01:00 2022] GET /identity/v3/users/neutron => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:00.571823 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd1eb7e-080f-47e8-9cec-7e6fe0ebe4ec None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:00.625450 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43163eb-7036-4c1c-97da-30a30201d057 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:00.659233 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:00.665807 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5455ace4-cb83-4d2d-971d-0581ef64f4ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:00.666488 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5455ace4-cb83-4d2d-971d-0581ef64f4ea None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:00.666766 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5455ace4-cb83-4d2d-971d-0581ef64f4ea None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:00.667042 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5455ace4-cb83-4d2d-971d-0581ef64f4ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:00.674907 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd1eb7e-080f-47e8-9cec-7e6fe0ebe4ec None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:00.676061 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:00.678166 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5455ace4-cb83-4d2d-971d-0581ef64f4ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:00.679656 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5455ace4-cb83-4d2d-971d-0581ef64f4ea None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:00.693519 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-5455ace4-cb83-4d2d-971d-0581ef64f4ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:00.693519 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:00.693519 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:00.693519 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:00.693519 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:00.693519 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:00.693519 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:00.693519 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:00.693519 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:00.693519 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:00.693519 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:00.693519 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:00.693519 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 03:01:00.693519 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 03:01:00.693519 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 03:01:00.693519 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 03:01:00.694663 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:00.694663 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:00.694663 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:00.694663 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:00.694663 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:00.694663 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:00.694663 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:00.694663 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:00.694663 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:00.694663 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:00.694663 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:00.694663 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:00.694663 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:00.694663 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:00.694663 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:00.694663 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:00.694663 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 03:01:00.695769 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 03:01:00.695769 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:00.695769 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:00.695769 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 257/518] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:01:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:00.695769 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5455ace4-cb83-4d2d-971d-0581ef64f4ea None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:00.711593 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e724832-94d9-4982-92c4-b18afff541a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:00.712207 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e724832-94d9-4982-92c4-b18afff541a8 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:00.712485 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e724832-94d9-4982-92c4-b18afff541a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:00.712755 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e724832-94d9-4982-92c4-b18afff541a8 None admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:00.713401 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e724832-94d9-4982-92c4-b18afff541a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:00.713703 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e724832-94d9-4982-92c4-b18afff541a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:00.715117 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e724832-94d9-4982-92c4-b18afff541a8 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:00.727451 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 262/519] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Aug 25 03:01:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:00.728062 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e724832-94d9-4982-92c4-b18afff541a8 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:00.795072 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5455ace4-cb83-4d2d-971d-0581ef64f4ea None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:00.825813 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e724832-94d9-4982-92c4-b18afff541a8 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:00.828572 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df11fe4-72f4-4972-bb9c-7b6967a8ed8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:00.829174 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df11fe4-72f4-4972-bb9c-7b6967a8ed8f None admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:00.829456 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df11fe4-72f4-4972-bb9c-7b6967a8ed8f None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:00.829729 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df11fe4-72f4-4972-bb9c-7b6967a8ed8f None admin] PATH_INFO: `/v3/users` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:00.830398 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df11fe4-72f4-4972-bb9c-7b6967a8ed8f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:00.831882 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df11fe4-72f4-4972-bb9c-7b6967a8ed8f None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:00.859306 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef50cc9-53a2-462b-9c8f-af42a765273e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:00.859912 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef50cc9-53a2-462b-9c8f-af42a765273e None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:00.861335 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef50cc9-53a2-462b-9c8f-af42a765273e None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:00.861335 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef50cc9-53a2-462b-9c8f-af42a765273e None admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:00.861335 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef50cc9-53a2-462b-9c8f-af42a765273e None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:00.861572 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef50cc9-53a2-462b-9c8f-af42a765273e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:00.862977 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef50cc9-53a2-462b-9c8f-af42a765273e None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:00.898852 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 263/520] 10.222.0.58 () {56 vars in 1137 bytes} [Thu Aug 25 03:01:00 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:00.899623 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef50cc9-53a2-462b-9c8f-af42a765273e None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:00.902146 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:00.912574 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 258/521] 10.222.0.58 () {60 vars in 1132 bytes} [Thu Aug 25 03:01:00 2022] POST /identity/v3/users => generated 247 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:01:00.917766 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df11fe4-72f4-4972-bb9c-7b6967a8ed8f None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:00.969693 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:01.028575 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df11fe4-72f4-4972-bb9c-7b6967a8ed8f None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:01.032352 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef50cc9-53a2-462b-9c8f-af42a765273e None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:01.074045 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936046f9-f708-4cc8-86e9-51ee7437c714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:01.075329 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936046f9-f708-4cc8-86e9-51ee7437c714 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:01.075329 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936046f9-f708-4cc8-86e9-51ee7437c714 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:01.075329 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936046f9-f708-4cc8-86e9-51ee7437c714 None admin] PATH_INFO: `/v3/domains` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:01.075843 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22de2760-c92a-4ead-a5ef-09598670e656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:01.075954 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936046f9-f708-4cc8-86e9-51ee7437c714 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:01.076161 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936046f9-f708-4cc8-86e9-51ee7437c714 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:01.076462 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22de2760-c92a-4ead-a5ef-09598670e656 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:01.078135 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22de2760-c92a-4ead-a5ef-09598670e656 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:01.078135 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22de2760-c92a-4ead-a5ef-09598670e656 None admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:01.078338 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936046f9-f708-4cc8-86e9-51ee7437c714 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:01.078411 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22de2760-c92a-4ead-a5ef-09598670e656 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:01.078675 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22de2760-c92a-4ead-a5ef-09598670e656 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:01.080281 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22de2760-c92a-4ead-a5ef-09598670e656 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:01.090631 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 259/522] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Aug 25 03:01:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:01.091464 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936046f9-f708-4cc8-86e9-51ee7437c714 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:01.103381 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 264/523] 10.222.0.58 () {56 vars in 1139 bytes} [Thu Aug 25 03:01:00 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:01.104224 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22de2760-c92a-4ead-a5ef-09598670e656 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:01.166622 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:01.203918 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936046f9-f708-4cc8-86e9-51ee7437c714 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:01.207904 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22de2760-c92a-4ead-a5ef-09598670e656 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:01.213439 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:01.241065 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3269c7c3-1151-499b-aab6-3a3842175625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:01.241711 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3269c7c3-1151-499b-aab6-3a3842175625 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:01.242099 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76888efe-bb77-46eb-82b2-cd059e02a196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:01.242355 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3269c7c3-1151-499b-aab6-3a3842175625 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:01.242630 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3269c7c3-1151-499b-aab6-3a3842175625 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:01.242808 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76888efe-bb77-46eb-82b2-cd059e02a196 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:01.243039 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76888efe-bb77-46eb-82b2-cd059e02a196 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:01.243314 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76888efe-bb77-46eb-82b2-cd059e02a196 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:01.251381 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3269c7c3-1151-499b-aab6-3a3842175625 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:01.253382 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3269c7c3-1151-499b-aab6-3a3842175625 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:01.270105 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-3269c7c3-1151-499b-aab6-3a3842175625 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 03:01:01.270105 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:01.270105 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:01.270105 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:01.270105 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:01.270105 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:01.270105 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:01.270105 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:01.270105 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:01.270105 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:01.270105 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:01.270105 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:01.270105 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 25 03:01:01.270105 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 25 03:01:01.270105 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 25 03:01:01.271202 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 25 03:01:01.271202 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:01.271202 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:01.271202 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:01.271202 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:01.271202 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:01.271202 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:01.271202 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:01.271202 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:01.271202 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:01.271202 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:01.271202 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:01.271202 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:01.271202 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:01.271202 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:01.271202 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:01.271202 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:01.276994 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 25 03:01:01.276994 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 25 03:01:01.276994 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 25 03:01:01.276994 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 25 03:01:01.276994 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 25 03:01:01.276994 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 25 03:01:01.276994 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 03:01:01.276994 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:01.276994 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 260/524] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Aug 25 03:01:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:01.276994 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3269c7c3-1151-499b-aab6-3a3842175625 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:01.314658 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76888efe-bb77-46eb-82b2-cd059e02a196 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:01.316114 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76888efe-bb77-46eb-82b2-cd059e02a196 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:01.383242 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:01.400928 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3269c7c3-1151-499b-aab6-3a3842175625 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:01.429918 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 265/525] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Aug 25 03:01:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 25 03:01:01.430624 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76888efe-bb77-46eb-82b2-cd059e02a196 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:01.438104 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9005240f-51cb-42da-882a-e51a7a0bb4f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:01.438774 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9005240f-51cb-42da-882a-e51a7a0bb4f9 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:01.439051 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9005240f-51cb-42da-882a-e51a7a0bb4f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:01.439335 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9005240f-51cb-42da-882a-e51a7a0bb4f9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:01.446080 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9005240f-51cb-42da-882a-e51a7a0bb4f9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:01.447624 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9005240f-51cb-42da-882a-e51a7a0bb4f9 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:01.459421 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-9005240f-51cb-42da-882a-e51a7a0bb4f9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 03:01:01.459421 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:01.459421 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:01.459421 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:01.459421 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:01.459421 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:01.459421 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:01.459421 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:01.459421 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:01.459421 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:01.459421 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:01.459421 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:01.459421 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 25 03:01:01.459421 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 25 03:01:01.459421 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 25 03:01:01.460365 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 25 03:01:01.460365 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:01.460365 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:01.460365 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:01.460365 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:01.460365 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:01.460365 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:01.460365 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:01.460365 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:01.460365 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:01.460365 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:01.460365 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:01.460365 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:01.460365 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:01.460365 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:01.460365 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:01.460365 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:01.461874 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 25 03:01:01.461874 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 25 03:01:01.461874 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 25 03:01:01.461874 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 25 03:01:01.461874 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 25 03:01:01.461874 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 25 03:01:01.461874 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 03:01:01.461874 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:01.461874 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 261/526] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Aug 25 03:01:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:01.462671 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9005240f-51cb-42da-882a-e51a7a0bb4f9 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:01.492669 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:01.543540 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76888efe-bb77-46eb-82b2-cd059e02a196 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:01.566294 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9005240f-51cb-42da-882a-e51a7a0bb4f9 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:01.586692 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae085b2-4021-4958-8535-c2bc014d87fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:01.587917 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae085b2-4021-4958-8535-c2bc014d87fe None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:01.588384 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae085b2-4021-4958-8535-c2bc014d87fe None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:01.588842 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae085b2-4021-4958-8535-c2bc014d87fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:01.604215 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a57be9-3ccb-4c4a-acec-a11575bebfb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:01.604823 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a57be9-3ccb-4c4a-acec-a11575bebfb8 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:01.605102 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a57be9-3ccb-4c4a-acec-a11575bebfb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:01.605375 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a57be9-3ccb-4c4a-acec-a11575bebfb8 None admin] PATH_INFO: `/v3/projects` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:01.606064 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a57be9-3ccb-4c4a-acec-a11575bebfb8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:01.606404 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a57be9-3ccb-4c4a-acec-a11575bebfb8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:01.607835 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a57be9-3ccb-4c4a-acec-a11575bebfb8 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:01.620602 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:01.631847 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 262/527] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Aug 25 03:01:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:01.631847 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a57be9-3ccb-4c4a-acec-a11575bebfb8 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:01.675085 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae085b2-4021-4958-8535-c2bc014d87fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:01.676989 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae085b2-4021-4958-8535-c2bc014d87fe None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:01.726448 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:01.739399 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a57be9-3ccb-4c4a-acec-a11575bebfb8 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:01.773901 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eb04ac-c370-4182-9aee-e4568085b14e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:01.774564 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eb04ac-c370-4182-9aee-e4568085b14e None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:01.774835 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eb04ac-c370-4182-9aee-e4568085b14e None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:01.775110 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eb04ac-c370-4182-9aee-e4568085b14e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:01.784956 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 266/528] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Aug 25 03:01:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 25 03:01:01.785978 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae085b2-4021-4958-8535-c2bc014d87fe None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:01.790266 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25eb04ac-c370-4182-9aee-e4568085b14e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:01.791759 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25eb04ac-c370-4182-9aee-e4568085b14e None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:01.808112 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-25eb04ac-c370-4182-9aee-e4568085b14e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:01.808112 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:01.808112 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:01.808112 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:01.808112 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:01.808112 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:01.808112 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:01.808112 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:01.808112 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:01.808112 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:01.808112 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:01.808112 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:01.808112 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 03:01:01.808112 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 03:01:01.808112 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 03:01:01.808112 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 03:01:01.809263 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:01.809263 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:01.809263 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:01.809263 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:01.809263 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:01.809263 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:01.809263 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:01.809263 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:01.809263 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:01.809263 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:01.809263 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:01.809263 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:01.809263 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:01.809263 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:01.809263 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:01.809263 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:01.809263 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 03:01:01.810363 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 03:01:01.810363 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:01.810363 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:01.814640 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 263/529] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:01:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:01.815094 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eb04ac-c370-4182-9aee-e4568085b14e None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:01.852827 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:01.893115 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae085b2-4021-4958-8535-c2bc014d87fe None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:01.920750 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eb04ac-c370-4182-9aee-e4568085b14e None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:01.929536 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed414dc-14e3-4335-a4fb-f4b2448e6f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:01.930476 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed414dc-14e3-4335-a4fb-f4b2448e6f15 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:01.930603 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed414dc-14e3-4335-a4fb-f4b2448e6f15 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:01.930827 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed414dc-14e3-4335-a4fb-f4b2448e6f15 None admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:01.931509 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed414dc-14e3-4335-a4fb-f4b2448e6f15 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:01.931919 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed414dc-14e3-4335-a4fb-f4b2448e6f15 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:01.933244 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed414dc-14e3-4335-a4fb-f4b2448e6f15 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:01.952860 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 267/530] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Aug 25 03:01:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:01.953732 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed414dc-14e3-4335-a4fb-f4b2448e6f15 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:01.962796 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccbc933-9711-4df2-a70a-93558e4f0a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:01.967957 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccbc933-9711-4df2-a70a-93558e4f0a30 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:01.967957 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccbc933-9711-4df2-a70a-93558e4f0a30 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:01.967957 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccbc933-9711-4df2-a70a-93558e4f0a30 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:01.967957 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccbc933-9711-4df2-a70a-93558e4f0a30 None admin] RBAC: Adding query filter params (role.id=a93ddb1fd6f6459fb46a17573b821422, scope.project.id=9fd4be43faec4e4191e0e4ec4a4588c3, user.id=0cd171bf4df94f8fab397e42835feeab) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:01.967957 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccbc933-9711-4df2-a70a-93558e4f0a30 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:01.967957 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccbc933-9711-4df2-a70a-93558e4f0a30 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:01.972807 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 264/531] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Aug 25 03:01:01 2022] GET /identity/v3/role_assignments?user.id=0cd171bf4df94f8fab397e42835feeab&scope.project.id=9fd4be43faec4e4191e0e4ec4a4588c3&role.id=a93ddb1fd6f6459fb46a17573b821422 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:01.973643 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccbc933-9711-4df2-a70a-93558e4f0a30 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:01.978639 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:02.095412 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccbc933-9711-4df2-a70a-93558e4f0a30 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:02.100881 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed414dc-14e3-4335-a4fb-f4b2448e6f15 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:02.131107 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:02.136716 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff10708-9900-4b83-8d82-1cd01d1cd8a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:02.137345 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff10708-9900-4b83-8d82-1cd01d1cd8a7 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:02.137632 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff10708-9900-4b83-8d82-1cd01d1cd8a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:02.137978 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff10708-9900-4b83-8d82-1cd01d1cd8a7 None admin] PATH_INFO: `/v3/domains` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:02.138789 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff10708-9900-4b83-8d82-1cd01d1cd8a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:02.139101 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff10708-9900-4b83-8d82-1cd01d1cd8a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:02.142546 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff10708-9900-4b83-8d82-1cd01d1cd8a7 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:02.153462 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d896d465-73ed-47cc-8f01-515a0229d502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:02.155345 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 265/532] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Aug 25 03:01:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:02.155504 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d896d465-73ed-47cc-8f01-515a0229d502 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:02.155504 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d896d465-73ed-47cc-8f01-515a0229d502 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:02.155647 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff10708-9900-4b83-8d82-1cd01d1cd8a7 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:02.155713 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d896d465-73ed-47cc-8f01-515a0229d502 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:02.168366 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d896d465-73ed-47cc-8f01-515a0229d502 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:02.171519 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d896d465-73ed-47cc-8f01-515a0229d502 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:02.192113 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-d896d465-73ed-47cc-8f01-515a0229d502 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:02.192113 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:02.192113 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:02.192113 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:02.192113 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:02.192113 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:02.192113 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:02.192113 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:02.192113 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:02.192113 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:02.192113 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:02.192113 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:02.192113 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 03:01:02.192113 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 03:01:02.192113 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 03:01:02.192113 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 03:01:02.193009 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:02.193009 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:02.193009 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:02.193009 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:02.193009 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:02.193009 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:02.193009 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:02.193009 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:02.193009 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:02.193009 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:02.193009 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:02.193009 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:02.193009 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:02.193009 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:02.193009 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:02.193009 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:02.193009 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 03:01:02.193831 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 03:01:02.193831 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:02.193831 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:02.196460 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 268/533] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:01:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:02.197497 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d896d465-73ed-47cc-8f01-515a0229d502 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:02.233877 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:02.274464 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff10708-9900-4b83-8d82-1cd01d1cd8a7 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:02.306105 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d896d465-73ed-47cc-8f01-515a0229d502 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:02.312892 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dfd7c3-54d2-4754-860c-fddc00176333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:02.313756 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dfd7c3-54d2-4754-860c-fddc00176333 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:02.314335 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dfd7c3-54d2-4754-860c-fddc00176333 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:02.314821 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dfd7c3-54d2-4754-860c-fddc00176333 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:02.315754 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dfd7c3-54d2-4754-860c-fddc00176333 None admin] RBAC: Adding query filter params (role.id=a93ddb1fd6f6459fb46a17573b821422, scope.project.id=9fd4be43faec4e4191e0e4ec4a4588c3, user.id=3f89089b390c4b6599e54000c0fa082a) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:02.316364 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dfd7c3-54d2-4754-860c-fddc00176333 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:02.318117 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dfd7c3-54d2-4754-860c-fddc00176333 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:02.330458 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 266/534] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Aug 25 03:01:02 2022] GET /identity/v3/role_assignments?user.id=3f89089b390c4b6599e54000c0fa082a&scope.project.id=9fd4be43faec4e4191e0e4ec4a4588c3&role.id=a93ddb1fd6f6459fb46a17573b821422 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:02.331956 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dfd7c3-54d2-4754-860c-fddc00176333 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:02.343808 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ee96b2-0a9e-47ca-a88e-873801699c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:02.344219 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ee96b2-0a9e-47ca-a88e-873801699c3d None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:02.350448 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ee96b2-0a9e-47ca-a88e-873801699c3d None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:02.350448 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ee96b2-0a9e-47ca-a88e-873801699c3d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:02.352550 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ee96b2-0a9e-47ca-a88e-873801699c3d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:02.354104 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ee96b2-0a9e-47ca-a88e-873801699c3d None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:02.367159 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-57ee96b2-0a9e-47ca-a88e-873801699c3d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 03:01:02.367159 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:02.367159 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:02.367159 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:02.367159 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:02.367159 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:02.367159 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:02.367159 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:02.367159 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:02.367159 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:02.367159 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:02.367159 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:02.367159 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 25 03:01:02.367159 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 25 03:01:02.367159 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 25 03:01:02.368108 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 25 03:01:02.368108 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:02.368108 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:02.368108 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:02.368108 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:02.368108 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:02.368108 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:02.368108 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:02.368108 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:02.368108 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:02.368108 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:02.368108 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:02.368108 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:02.368108 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:02.368108 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:02.368108 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:02.368108 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:02.369485 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 25 03:01:02.369485 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 25 03:01:02.369485 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 25 03:01:02.369485 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 25 03:01:02.369485 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 25 03:01:02.369485 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 25 03:01:02.369485 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 03:01:02.369485 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:02.369485 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 269/535] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Aug 25 03:01:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:02.378318 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ee96b2-0a9e-47ca-a88e-873801699c3d None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:02.447825 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:02.537806 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ee96b2-0a9e-47ca-a88e-873801699c3d None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:02.546849 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dfd7c3-54d2-4754-860c-fddc00176333 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:02.576853 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:02.580820 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46c879c-4ce3-4d66-b1f3-5322c647c892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:02.581760 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46c879c-4ce3-4d66-b1f3-5322c647c892 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:02.582237 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46c879c-4ce3-4d66-b1f3-5322c647c892 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:02.582858 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46c879c-4ce3-4d66-b1f3-5322c647c892 None admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:02.583748 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46c879c-4ce3-4d66-b1f3-5322c647c892 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:02.584246 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46c879c-4ce3-4d66-b1f3-5322c647c892 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:02.586030 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46c879c-4ce3-4d66-b1f3-5322c647c892 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:02.590608 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5a7c41-0ddc-41a8-847c-fd49f80e2ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:02.591230 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5a7c41-0ddc-41a8-847c-fd49f80e2ad0 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:02.591512 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5a7c41-0ddc-41a8-847c-fd49f80e2ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:02.591787 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5a7c41-0ddc-41a8-847c-fd49f80e2ad0 None admin] PATH_INFO: `/v3/domains` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:02.593561 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5a7c41-0ddc-41a8-847c-fd49f80e2ad0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:02.593561 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5a7c41-0ddc-41a8-847c-fd49f80e2ad0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:02.595406 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5a7c41-0ddc-41a8-847c-fd49f80e2ad0 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:02.600910 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 270/536] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Aug 25 03:01:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:01:02.609192 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46c879c-4ce3-4d66-b1f3-5322c647c892 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:02.618057 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 267/537] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Aug 25 03:01:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:02.619290 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5a7c41-0ddc-41a8-847c-fd49f80e2ad0 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:02.787170 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:02.805649 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46c879c-4ce3-4d66-b1f3-5322c647c892 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:02.832256 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5a7c41-0ddc-41a8-847c-fd49f80e2ad0 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:02.840042 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bf5362-4f89-4997-be2c-94ded61a9498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:02.840667 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bf5362-4f89-4997-be2c-94ded61a9498 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:02.840947 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bf5362-4f89-4997-be2c-94ded61a9498 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:02.841220 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bf5362-4f89-4997-be2c-94ded61a9498 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:02.841957 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bf5362-4f89-4997-be2c-94ded61a9498 None admin] RBAC: Adding query filter params (role.id=a93ddb1fd6f6459fb46a17573b821422, scope.project.id=9fd4be43faec4e4191e0e4ec4a4588c3, user.id=60193ba9e1c14df3bc9eebcc6f28f2d7) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:02.842263 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bf5362-4f89-4997-be2c-94ded61a9498 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:02.843773 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bf5362-4f89-4997-be2c-94ded61a9498 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:02.850437 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 271/538] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Aug 25 03:01:02 2022] GET /identity/v3/role_assignments?user.id=60193ba9e1c14df3bc9eebcc6f28f2d7&scope.project.id=9fd4be43faec4e4191e0e4ec4a4588c3&role.id=a93ddb1fd6f6459fb46a17573b821422 => generated 259 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:01:02.882519 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894964c0-ded9-4ca9-a521-04a41a72b8a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:02.883235 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894964c0-ded9-4ca9-a521-04a41a72b8a5 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:02.883523 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894964c0-ded9-4ca9-a521-04a41a72b8a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:02.883803 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894964c0-ded9-4ca9-a521-04a41a72b8a5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:02.891787 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894964c0-ded9-4ca9-a521-04a41a72b8a5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:02.893305 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894964c0-ded9-4ca9-a521-04a41a72b8a5 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:02.908784 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-894964c0-ded9-4ca9-a521-04a41a72b8a5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 03:01:02.908784 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:02.908784 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:02.908784 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:02.908784 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:02.908784 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:02.908784 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:02.908784 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:02.908784 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:02.908784 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:02.908784 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:02.908784 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:02.908784 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 25 03:01:02.908784 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 25 03:01:02.908784 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 25 03:01:02.909684 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 25 03:01:02.909684 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:02.909684 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:02.909684 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:02.909684 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:02.909684 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:02.909684 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:02.909684 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:02.909684 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:02.909684 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:02.909684 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:02.909684 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:02.909684 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:02.909684 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:02.909684 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:02.909684 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:02.909684 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:02.910571 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 25 03:01:02.910571 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 25 03:01:02.910571 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 25 03:01:02.910571 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 25 03:01:02.910571 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 25 03:01:02.910571 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 25 03:01:02.910571 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 03:01:02.910571 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:02.910571 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 268/539] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Aug 25 03:01:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:02.916077 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bf5362-4f89-4997-be2c-94ded61a9498 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:02.971880 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e8f16b-3c33-4414-86e3-53fdd46d24b2 None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:02.972209 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e8f16b-3c33-4414-86e3-53fdd46d24b2 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:02.972474 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e8f16b-3c33-4414-86e3-53fdd46d24b2 None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:02.975272 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 269/540] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:01:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:01:02.979906 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c9b35d-f61f-437f-87e4-0efbfafe3f9a None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:02.980182 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c9b35d-f61f-437f-87e4-0efbfafe3f9a None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:02.980460 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c9b35d-f61f-437f-87e4-0efbfafe3f9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:03.013856 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-74c9b35d-f61f-437f-87e4-0efbfafe3f9a None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:01:03.015146 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-74c9b35d-f61f-437f-87e4-0efbfafe3f9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:01:03.058215 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bf5362-4f89-4997-be2c-94ded61a9498 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:03.074623 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:03.115134 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 270/541] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:01:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 25 03:01:03.141473 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e2f085-2270-410b-9523-a1f5efc76b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:03.142201 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e2f085-2270-410b-9523-a1f5efc76b98 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:03.142523 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e2f085-2270-410b-9523-a1f5efc76b98 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:03.142829 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e2f085-2270-410b-9523-a1f5efc76b98 None admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:03.143534 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e2f085-2270-410b-9523-a1f5efc76b98 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:03.143839 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e2f085-2270-410b-9523-a1f5efc76b98 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:03.145331 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e2f085-2270-410b-9523-a1f5efc76b98 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:03.168388 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 272/542] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Aug 25 03:01:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:03.174827 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c9b35d-f61f-437f-87e4-0efbfafe3f9a None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:03.323446 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c9b35d-f61f-437f-87e4-0efbfafe3f9a None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:03.361462 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b12f98-9a53-416b-92a4-9124fcb7b136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:03.362083 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b12f98-9a53-416b-92a4-9124fcb7b136 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:03.362403 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b12f98-9a53-416b-92a4-9124fcb7b136 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:03.362682 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b12f98-9a53-416b-92a4-9124fcb7b136 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:03.363425 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b12f98-9a53-416b-92a4-9124fcb7b136 None admin] RBAC: Adding query filter params (role.id=a93ddb1fd6f6459fb46a17573b821422, scope.project.id=9fd4be43faec4e4191e0e4ec4a4588c3, user.id=a0a1a7f091774842907fd08de15e8058) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:03.363738 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b12f98-9a53-416b-92a4-9124fcb7b136 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:03.365196 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b12f98-9a53-416b-92a4-9124fcb7b136 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:03.371360 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 271/543] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Aug 25 03:01:03 2022] GET /identity/v3/role_assignments?user.id=a0a1a7f091774842907fd08de15e8058&scope.project.id=9fd4be43faec4e4191e0e4ec4a4588c3&role.id=a93ddb1fd6f6459fb46a17573b821422 => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:03.496422 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e2f085-2270-410b-9523-a1f5efc76b98 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:03.515021 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:03.621376 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e2f085-2270-410b-9523-a1f5efc76b98 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:03.658652 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3299b3b-69a4-4498-94c8-6cbee329af95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:03.659299 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3299b3b-69a4-4498-94c8-6cbee329af95 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:03.659579 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3299b3b-69a4-4498-94c8-6cbee329af95 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:03.659850 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3299b3b-69a4-4498-94c8-6cbee329af95 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:03.670672 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3299b3b-69a4-4498-94c8-6cbee329af95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:01:03.675305 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-f3299b3b-69a4-4498-94c8-6cbee329af95 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 25 03:01:03.675305 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:03.675305 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:03.675305 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:03.675305 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:03.675305 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:03.675305 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:03.675305 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:03.675305 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:03.675305 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:03.675305 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:03.675305 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:03.675305 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:01:03.675305 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:01:03.675305 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:01:03.675305 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:01:03.676153 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:01:03.676153 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:01:03.676153 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:01:03.676153 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:01:03.676153 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:03.676153 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:03.676153 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:03.676153 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:03.676153 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:03.676153 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:03.676153 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:03.676153 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:03.676153 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:03.676153 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:03.676153 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:03.676153 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:03.676939 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:03.676939 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:03.676939 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:03.676939 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:03.676939 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:01:03.676939 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:01:03.676939 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:01:03.676939 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:01:03.676939 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:01:03.676939 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:01:03.676939 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 25 03:01:03.676939 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:03.680611 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 273/544] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Aug 25 03:01:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:03.682674 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b12f98-9a53-416b-92a4-9124fcb7b136 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:03.739816 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:03.810446 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b12f98-9a53-416b-92a4-9124fcb7b136 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:03.856333 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b90134-d63a-48e6-8eef-05ceaf342405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:03.856955 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b90134-d63a-48e6-8eef-05ceaf342405 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:03.857236 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b90134-d63a-48e6-8eef-05ceaf342405 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:03.857508 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b90134-d63a-48e6-8eef-05ceaf342405 None admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:03.858213 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b90134-d63a-48e6-8eef-05ceaf342405 None admin] RBAC: Adding query filter params (name=service) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:03.858539 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b90134-d63a-48e6-8eef-05ceaf342405 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:03.859979 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b90134-d63a-48e6-8eef-05ceaf342405 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:03.869602 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 272/545] 10.222.0.58 () {56 vars in 1103 bytes} [Thu Aug 25 03:01:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:03.875400 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3299b3b-69a4-4498-94c8-6cbee329af95 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:03.963106 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:03.994843 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3299b3b-69a4-4498-94c8-6cbee329af95 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:04.041906 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7132d2-804b-42df-b6ba-2cb2b1f7c8d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:04.042583 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7132d2-804b-42df-b6ba-2cb2b1f7c8d5 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:04.042866 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7132d2-804b-42df-b6ba-2cb2b1f7c8d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:04.043145 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7132d2-804b-42df-b6ba-2cb2b1f7c8d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:04.054599 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7132d2-804b-42df-b6ba-2cb2b1f7c8d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:04.056076 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7132d2-804b-42df-b6ba-2cb2b1f7c8d5 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:04.081513 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-fa7132d2-804b-42df-b6ba-2cb2b1f7c8d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:04.081513 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:04.081513 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:04.081513 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:04.081513 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:04.081513 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:04.081513 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:04.081513 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:04.081513 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:04.081513 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:04.081513 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:04.081513 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:04.081513 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 03:01:04.081513 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 03:01:04.081513 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 03:01:04.081513 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 03:01:04.083758 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:04.083758 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:04.083758 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:04.083758 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:04.083758 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:04.083758 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:04.083758 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:04.083758 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:04.083758 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:04.083758 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:04.083758 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:04.083758 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:04.083758 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:04.083758 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:04.083758 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:04.083758 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:04.083758 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 03:01:04.085905 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 03:01:04.085905 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:04.085905 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:04.085905 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 274/546] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:01:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 03:01:04.088780 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b90134-d63a-48e6-8eef-05ceaf342405 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:04.245076 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b90134-d63a-48e6-8eef-05ceaf342405 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:04.250194 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:04.290368 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d520d820-5667-4c56-9c78-7349c2977919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:04.290998 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d520d820-5667-4c56-9c78-7349c2977919 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:04.291280 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d520d820-5667-4c56-9c78-7349c2977919 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:04.291554 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d520d820-5667-4c56-9c78-7349c2977919 None admin] PATH_INFO: `/v3/domains` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:04.292214 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d520d820-5667-4c56-9c78-7349c2977919 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:04.292519 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d520d820-5667-4c56-9c78-7349c2977919 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:04.293895 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d520d820-5667-4c56-9c78-7349c2977919 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:04.310124 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 273/547] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Aug 25 03:01:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:04.315272 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7132d2-804b-42df-b6ba-2cb2b1f7c8d5 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:04.413800 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2524b2a-4ec8-4fd8-bc8a-f1eb749a21ce None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:04.414166 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2524b2a-4ec8-4fd8-bc8a-f1eb749a21ce None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:04.414383 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2524b2a-4ec8-4fd8-bc8a-f1eb749a21ce None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:04.415906 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 274/548] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:01:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:01:04.424839 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c4af62-2c1f-4bdf-8e64-16a8d28627c8 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:04.425330 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c4af62-2c1f-4bdf-8e64-16a8d28627c8 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:04.425797 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c4af62-2c1f-4bdf-8e64-16a8d28627c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:04.458007 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-a9c4af62-2c1f-4bdf-8e64-16a8d28627c8 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:01:04.459415 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-a9c4af62-2c1f-4bdf-8e64-16a8d28627c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:01:04.464592 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7132d2-804b-42df-b6ba-2cb2b1f7c8d5 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:04.515708 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:04.531156 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb38b35-a175-4bf7-99ea-39c11e6c4395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:04.540363 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb38b35-a175-4bf7-99ea-39c11e6c4395 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:04.540363 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb38b35-a175-4bf7-99ea-39c11e6c4395 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:04.540363 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb38b35-a175-4bf7-99ea-39c11e6c4395 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:04.540687 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 275/549] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:01:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 03:01:04.547315 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb38b35-a175-4bf7-99ea-39c11e6c4395 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:04.549038 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb38b35-a175-4bf7-99ea-39c11e6c4395 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:04.560586 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-adb38b35-a175-4bf7-99ea-39c11e6c4395 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 25 03:01:04.560586 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:04.560586 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:04.560586 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:04.560586 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:04.560586 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:04.560586 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:04.560586 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:04.560586 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:04.560586 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:04.560586 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:04.560586 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:04.560586 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 25 03:01:04.560586 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 25 03:01:04.560586 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 25 03:01:04.560586 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 25 03:01:04.561535 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:04.561535 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:04.561535 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 25 03:01:04.561535 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 03:01:04.561535 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 25 03:01:04.561535 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 03:01:04.561535 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:04.561535 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:04.561535 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:04.561535 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:04.561535 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:04.561535 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:04.561535 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:04.561535 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:04.561535 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:04.561535 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:04.561535 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:04.561535 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:04.562478 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:04.562478 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:04.562478 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 25 03:01:04.562478 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 25 03:01:04.562478 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 25 03:01:04.562478 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 25 03:01:04.562478 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 25 03:01:04.562478 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 25 03:01:04.562478 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 25 03:01:04.562478 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:04.565251 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 275/550] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Aug 25 03:01:04 2022] GET /identity/v3/users/swift => generated 83 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:04.570945 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c4af62-2c1f-4bdf-8e64-16a8d28627c8 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:04.667060 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf012b13-abb0-4009-b831-28286b0707ae None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:04.667370 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf012b13-abb0-4009-b831-28286b0707ae None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:04.667618 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf012b13-abb0-4009-b831-28286b0707ae None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:04.670626 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 276/551] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:01:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:01:04.676012 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44c76e0-6a9a-4003-950a-c48f9f3e544a None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:04.676287 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44c76e0-6a9a-4003-950a-c48f9f3e544a None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:04.676567 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44c76e0-6a9a-4003-950a-c48f9f3e544a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:04.707739 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-e44c76e0-6a9a-4003-950a-c48f9f3e544a None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:01:04.708917 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-e44c76e0-6a9a-4003-950a-c48f9f3e544a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:01:04.710644 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:04.717238 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c4af62-2c1f-4bdf-8e64-16a8d28627c8 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:04.782090 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde48b4b-d8b7-43cc-b383-86a44619e969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:04.783432 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde48b4b-d8b7-43cc-b383-86a44619e969 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:04.784781 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde48b4b-d8b7-43cc-b383-86a44619e969 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:04.785743 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde48b4b-d8b7-43cc-b383-86a44619e969 None admin] PATH_INFO: `/v3/users` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:04.787328 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde48b4b-d8b7-43cc-b383-86a44619e969 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:04.788756 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde48b4b-d8b7-43cc-b383-86a44619e969 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:04.790362 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde48b4b-d8b7-43cc-b383-86a44619e969 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:04.831033 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 276/552] 10.222.0.58 () {56 vars in 1135 bytes} [Thu Aug 25 03:01:04 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:04.836466 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde48b4b-d8b7-43cc-b383-86a44619e969 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:04.849085 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 277/553] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:01:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 03:01:04.916917 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44c76e0-6a9a-4003-950a-c48f9f3e544a None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:04.986025 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde48b4b-d8b7-43cc-b383-86a44619e969 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:05.064762 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd69349-e19f-42af-8895-8fd8c4a8aef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:05.065712 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd69349-e19f-42af-8895-8fd8c4a8aef3 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:05.066206 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd69349-e19f-42af-8895-8fd8c4a8aef3 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:05.066707 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd69349-e19f-42af-8895-8fd8c4a8aef3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:05.069572 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44c76e0-6a9a-4003-950a-c48f9f3e544a None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:05.103923 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:05.104505 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:05.135920 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678649c6-315e-4fb0-b604-9e4cf5ae240d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:05.136558 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678649c6-315e-4fb0-b604-9e4cf5ae240d None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:05.136830 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678649c6-315e-4fb0-b604-9e4cf5ae240d None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:05.137098 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678649c6-315e-4fb0-b604-9e4cf5ae240d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:05.157557 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678649c6-315e-4fb0-b604-9e4cf5ae240d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:01:05.165233 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-678649c6-315e-4fb0-b604-9e4cf5ae240d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 25 03:01:05.165233 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:05.165233 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:05.165233 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:05.165233 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:05.165233 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:05.165233 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:05.165233 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:05.165233 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:05.165233 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:05.165233 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:05.165233 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:05.165233 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:01:05.165233 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:01:05.165233 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:01:05.165233 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:01:05.166105 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:01:05.166105 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:01:05.166105 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:01:05.166105 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:01:05.166105 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:05.166105 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:05.166105 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:05.166105 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:05.166105 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:05.166105 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:05.166105 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:05.166105 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:05.166105 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:05.166105 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:05.166105 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:05.166105 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:05.167032 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:05.167032 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:05.167032 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:05.167032 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:05.167032 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:01:05.167032 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:01:05.167032 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:01:05.167032 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:01:05.167032 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:01:05.167032 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:01:05.167032 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 25 03:01:05.167032 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:05.167032 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 278/554] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Aug 25 03:01:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 03:01:05.172946 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678649c6-315e-4fb0-b604-9e4cf5ae240d None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:05.194979 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd69349-e19f-42af-8895-8fd8c4a8aef3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:05.196498 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd69349-e19f-42af-8895-8fd8c4a8aef3 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:05.297555 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 277/555] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Aug 25 03:01:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 25 03:01:05.298690 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0385ec5a-399c-407f-8fba-6a7f8d8bce8e None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:05.298972 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0385ec5a-399c-407f-8fba-6a7f8d8bce8e None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:05.299248 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0385ec5a-399c-407f-8fba-6a7f8d8bce8e None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:05.300728 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 278/556] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:01:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:01:05.301425 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0385ec5a-399c-407f-8fba-6a7f8d8bce8e None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:05.302994 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678649c6-315e-4fb0-b604-9e4cf5ae240d None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:05.341452 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c43f334-de5f-4ec2-9281-18de5bc13fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:05.342561 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c43f334-de5f-4ec2-9281-18de5bc13fd8 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:05.344256 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c43f334-de5f-4ec2-9281-18de5bc13fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:05.344722 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c43f334-de5f-4ec2-9281-18de5bc13fd8 None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:05.345520 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c43f334-de5f-4ec2-9281-18de5bc13fd8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:05.345963 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c43f334-de5f-4ec2-9281-18de5bc13fd8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:05.347500 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c43f334-de5f-4ec2-9281-18de5bc13fd8 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:05.360570 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:05.363138 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 279/557] 10.222.0.58 () {56 vars in 1103 bytes} [Thu Aug 25 03:01:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:05.364921 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c43f334-de5f-4ec2-9281-18de5bc13fd8 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:05.410718 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:05.412432 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0385ec5a-399c-407f-8fba-6a7f8d8bce8e None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:05.450270 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0deb91-ca87-4d2b-a513-704675feb9e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:05.450955 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0deb91-ca87-4d2b-a513-704675feb9e6 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:05.451237 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0deb91-ca87-4d2b-a513-704675feb9e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:05.451547 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0deb91-ca87-4d2b-a513-704675feb9e6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:05.463218 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0deb91-ca87-4d2b-a513-704675feb9e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:01:05.468923 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-de0deb91-ca87-4d2b-a513-704675feb9e6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 25 03:01:05.468923 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:05.468923 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:05.468923 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:05.468923 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:05.468923 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:05.468923 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:05.468923 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:05.468923 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:05.468923 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:05.468923 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:05.468923 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:05.468923 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:01:05.468923 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:01:05.468923 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:01:05.468923 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:01:05.470091 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:01:05.470091 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:01:05.470091 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:01:05.470091 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:01:05.470091 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:05.470091 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:05.470091 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:05.470091 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:05.470091 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:05.470091 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:05.470091 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:05.470091 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:05.470091 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:05.470091 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:05.470091 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:05.470091 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:05.471059 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:05.471059 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:05.471059 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:05.471059 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:05.471059 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:01:05.471059 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:01:05.471059 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:01:05.471059 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:01:05.471059 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:01:05.471059 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:01:05.471059 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 25 03:01:05.471059 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:05.471059 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 279/558] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Aug 25 03:01:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:05.475811 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3443591-6919-4fec-a836-b3b9d9f2f130 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:05.476096 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3443591-6919-4fec-a836-b3b9d9f2f130 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:05.476374 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3443591-6919-4fec-a836-b3b9d9f2f130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:05.477676 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c43f334-de5f-4ec2-9281-18de5bc13fd8 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:05.493573 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-a3443591-6919-4fec-a836-b3b9d9f2f130 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:01:05.494772 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-a3443591-6919-4fec-a836-b3b9d9f2f130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:01:05.513173 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cde095-c1d5-435f-a955-3f322f4d6247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:05.514006 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cde095-c1d5-435f-a955-3f322f4d6247 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:05.515102 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cde095-c1d5-435f-a955-3f322f4d6247 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:05.515550 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cde095-c1d5-435f-a955-3f322f4d6247 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:05.526334 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cde095-c1d5-435f-a955-3f322f4d6247 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:05.527758 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cde095-c1d5-435f-a955-3f322f4d6247 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:05.541876 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-15cde095-c1d5-435f-a955-3f322f4d6247 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:05.541876 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:05.541876 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:05.541876 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:05.541876 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:05.541876 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:05.541876 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:05.541876 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:05.541876 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:05.541876 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:05.541876 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:05.541876 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:05.541876 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 03:01:05.541876 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 03:01:05.541876 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 03:01:05.541876 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 03:01:05.542885 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:05.542885 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:05.542885 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:05.542885 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:05.542885 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:05.542885 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:05.542885 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:05.542885 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:05.542885 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:05.542885 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:05.542885 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:05.542885 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:05.542885 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:05.542885 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:05.542885 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:05.542885 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:05.542885 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 03:01:05.543767 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 03:01:05.543767 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:05.543767 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:05.543767 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 280/559] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:01:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:05.543767 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cde095-c1d5-435f-a955-3f322f4d6247 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:05.578306 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 280/560] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:01:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 03:01:05.578306 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3443591-6919-4fec-a836-b3b9d9f2f130 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:05.634958 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:05.669496 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:05.681675 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cde095-c1d5-435f-a955-3f322f4d6247 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:05.696948 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3443591-6919-4fec-a836-b3b9d9f2f130 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:05.727963 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bf8b2c-b7ee-4934-bc03-68f107e9ae6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:05.729502 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bf8b2c-b7ee-4934-bc03-68f107e9ae6a None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:05.730080 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bf8b2c-b7ee-4934-bc03-68f107e9ae6a None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:05.730689 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bf8b2c-b7ee-4934-bc03-68f107e9ae6a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:05.734799 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730b30ff-4212-4148-8571-0b7c979f6fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:05.735421 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730b30ff-4212-4148-8571-0b7c979f6fe6 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:05.735830 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730b30ff-4212-4148-8571-0b7c979f6fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:05.736174 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730b30ff-4212-4148-8571-0b7c979f6fe6 None admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:05.736914 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730b30ff-4212-4148-8571-0b7c979f6fe6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:05.737337 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730b30ff-4212-4148-8571-0b7c979f6fe6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:05.738898 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730b30ff-4212-4148-8571-0b7c979f6fe6 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:05.744363 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bf8b2c-b7ee-4934-bc03-68f107e9ae6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:05.746728 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bf8b2c-b7ee-4934-bc03-68f107e9ae6a None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:05.750437 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 281/561] 10.222.0.58 () {56 vars in 1103 bytes} [Thu Aug 25 03:01:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:05.751557 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730b30ff-4212-4148-8571-0b7c979f6fe6 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:05.771975 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-c7bf8b2c-b7ee-4934-bc03-68f107e9ae6a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:05.771975 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:05.771975 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:05.771975 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:05.771975 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:05.771975 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:05.771975 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:05.771975 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:05.771975 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:05.771975 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:05.771975 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:05.771975 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:05.771975 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 03:01:05.771975 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 03:01:05.771975 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 03:01:05.771975 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 03:01:05.772933 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:05.772933 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:05.772933 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:05.772933 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:05.772933 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:05.772933 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:05.772933 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:05.772933 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:05.772933 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:05.772933 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:05.772933 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:05.772933 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:05.772933 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:05.772933 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:05.772933 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:05.772933 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:05.772933 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 03:01:05.773873 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 03:01:05.773873 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:05.773873 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:05.780414 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 281/562] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:01:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:05.780414 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bf8b2c-b7ee-4934-bc03-68f107e9ae6a None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:05.871055 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730b30ff-4212-4148-8571-0b7c979f6fe6 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:05.873698 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:05.883983 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bf8b2c-b7ee-4934-bc03-68f107e9ae6a None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:05.904411 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2529bff0-15be-4c70-8456-c4dec85e009d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:05.905031 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2529bff0-15be-4c70-8456-c4dec85e009d None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:05.905312 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2529bff0-15be-4c70-8456-c4dec85e009d None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:05.905584 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2529bff0-15be-4c70-8456-c4dec85e009d None admin] PATH_INFO: `/v3/domains` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:05.906225 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2529bff0-15be-4c70-8456-c4dec85e009d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:05.906558 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2529bff0-15be-4c70-8456-c4dec85e009d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:05.908122 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2529bff0-15be-4c70-8456-c4dec85e009d None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:05.916517 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245d5ac7-1ca1-461c-8033-722da5ab3063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:05.917139 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245d5ac7-1ca1-461c-8033-722da5ab3063 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:05.917417 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245d5ac7-1ca1-461c-8033-722da5ab3063 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:05.917690 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245d5ac7-1ca1-461c-8033-722da5ab3063 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:05.919484 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 282/563] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Aug 25 03:01:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:05.920313 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2529bff0-15be-4c70-8456-c4dec85e009d None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:05.928503 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245d5ac7-1ca1-461c-8033-722da5ab3063 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:05.929945 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245d5ac7-1ca1-461c-8033-722da5ab3063 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:05.932446 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:05.944947 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-245d5ac7-1ca1-461c-8033-722da5ab3063 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:05.944947 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:05.944947 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:05.944947 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:05.944947 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:05.944947 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:05.944947 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:05.944947 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:05.944947 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:05.944947 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:05.944947 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:05.944947 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:05.944947 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 03:01:05.944947 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 03:01:05.944947 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 03:01:05.944947 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 03:01:05.946896 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:05.946896 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:05.946896 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:05.946896 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:05.946896 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:05.946896 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:05.946896 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:05.946896 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:05.946896 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:05.946896 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:05.946896 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:05.946896 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:05.946896 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:05.946896 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:05.946896 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:05.946896 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:05.946896 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 03:01:05.948032 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 03:01:05.948032 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:05.948032 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:05.948032 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 282/564] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:01:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:05.948032 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245d5ac7-1ca1-461c-8033-722da5ab3063 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:06.022304 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2529bff0-15be-4c70-8456-c4dec85e009d None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:06.045419 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245d5ac7-1ca1-461c-8033-722da5ab3063 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:06.054683 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fade967-12e7-478e-b0f9-4ae9f4de6a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:06.055285 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fade967-12e7-478e-b0f9-4ae9f4de6a4d None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:06.055570 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fade967-12e7-478e-b0f9-4ae9f4de6a4d None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:06.055844 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fade967-12e7-478e-b0f9-4ae9f4de6a4d None admin] PATH_INFO: `/v3/domains` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:06.056485 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fade967-12e7-478e-b0f9-4ae9f4de6a4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:06.056792 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fade967-12e7-478e-b0f9-4ae9f4de6a4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:06.058152 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fade967-12e7-478e-b0f9-4ae9f4de6a4d None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:06.070127 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 283/565] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Aug 25 03:01:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:06.071340 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5401fd2a-448e-4d5e-868c-8d27b7b491ce None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:06.071626 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5401fd2a-448e-4d5e-868c-8d27b7b491ce None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:06.071907 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5401fd2a-448e-4d5e-868c-8d27b7b491ce None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:06.077132 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 284/566] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:01:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:01:06.078148 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5401fd2a-448e-4d5e-868c-8d27b7b491ce None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:06.080432 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb59d8f-4a06-4e83-96db-f6c32c2adad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:06.081061 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb59d8f-4a06-4e83-96db-f6c32c2adad0 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:06.081339 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb59d8f-4a06-4e83-96db-f6c32c2adad0 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:06.081616 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb59d8f-4a06-4e83-96db-f6c32c2adad0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:06.091904 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb59d8f-4a06-4e83-96db-f6c32c2adad0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:01:06.097173 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-ceb59d8f-4a06-4e83-96db-f6c32c2adad0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 25 03:01:06.097173 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:06.097173 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:06.097173 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:06.097173 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:06.097173 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:06.097173 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:06.097173 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:06.097173 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:06.097173 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:06.097173 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:06.097173 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:06.097173 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:01:06.097173 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:01:06.097173 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:01:06.097173 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:01:06.098340 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:01:06.098340 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:01:06.098340 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:01:06.098340 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:01:06.098340 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:06.098340 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:06.098340 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:06.098340 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:06.098340 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:06.098340 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:06.098340 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:06.098340 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:06.098340 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:06.098340 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:06.098340 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:06.098340 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:06.099194 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:06.099194 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:06.099194 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:06.099194 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:06.099194 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:01:06.099194 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:01:06.099194 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:01:06.099194 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:01:06.099194 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:01:06.099194 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:01:06.099194 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 25 03:01:06.099194 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:06.099194 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 283/567] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Aug 25 03:01:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:06.099900 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb59d8f-4a06-4e83-96db-f6c32c2adad0 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:06.103540 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:06.154237 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:06.178269 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5401fd2a-448e-4d5e-868c-8d27b7b491ce None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:06.197811 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb59d8f-4a06-4e83-96db-f6c32c2adad0 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:06.211767 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15fecc8-5114-4e51-be0d-9e3a95b41d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:06.212385 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15fecc8-5114-4e51-be0d-9e3a95b41d5e None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:06.212665 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15fecc8-5114-4e51-be0d-9e3a95b41d5e None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:06.212939 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15fecc8-5114-4e51-be0d-9e3a95b41d5e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:06.219368 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15fecc8-5114-4e51-be0d-9e3a95b41d5e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:06.220822 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15fecc8-5114-4e51-be0d-9e3a95b41d5e None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:06.230047 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51454dcf-cb1c-4946-995c-2e8a3c3be1ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:06.230680 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51454dcf-cb1c-4946-995c-2e8a3c3be1ad None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:06.230964 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51454dcf-cb1c-4946-995c-2e8a3c3be1ad None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:06.231239 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51454dcf-cb1c-4946-995c-2e8a3c3be1ad None admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:06.231879 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51454dcf-cb1c-4946-995c-2e8a3c3be1ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:06.232182 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51454dcf-cb1c-4946-995c-2e8a3c3be1ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:06.232289 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-f15fecc8-5114-4e51-be0d-9e3a95b41d5e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 03:01:06.232289 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:06.232289 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:06.232289 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:06.232289 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:06.232289 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:06.232289 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:06.232289 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:06.232289 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:06.232289 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:06.232289 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:06.232289 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:06.232289 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 25 03:01:06.232289 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 25 03:01:06.232289 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 25 03:01:06.233764 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 25 03:01:06.233764 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:06.233764 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:06.233764 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:06.233764 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:06.233764 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:06.233764 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:06.233764 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:06.233764 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:06.233764 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:06.233764 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:06.233764 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:06.233764 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:06.233764 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:06.233764 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:06.233764 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:06.233764 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:06.234708 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 25 03:01:06.234708 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 25 03:01:06.234708 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 25 03:01:06.234708 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 25 03:01:06.234708 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 25 03:01:06.234708 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 25 03:01:06.234708 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 03:01:06.234708 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:06.235135 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51454dcf-cb1c-4946-995c-2e8a3c3be1ad None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:06.235298 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 285/568] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Aug 25 03:01:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:06.235298 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15fecc8-5114-4e51-be0d-9e3a95b41d5e None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:06.249779 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 284/569] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Aug 25 03:01:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:06.249779 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3127f5aa-dd20-4df9-98a8-43eb4d049408 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:06.249779 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3127f5aa-dd20-4df9-98a8-43eb4d049408 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:06.249779 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3127f5aa-dd20-4df9-98a8-43eb4d049408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:06.264954 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-3127f5aa-dd20-4df9-98a8-43eb4d049408 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:01:06.266135 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-3127f5aa-dd20-4df9-98a8-43eb4d049408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:01:06.320107 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:06.331187 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 285/570] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:01:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 03:01:06.335645 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3127f5aa-dd20-4df9-98a8-43eb4d049408 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:06.335756 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15fecc8-5114-4e51-be0d-9e3a95b41d5e None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:06.371590 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c38941-d65f-4ec0-99a8-28d1444d5f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:06.372219 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c38941-d65f-4ec0-99a8-28d1444d5f7d None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:06.372502 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c38941-d65f-4ec0-99a8-28d1444d5f7d None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:06.372780 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c38941-d65f-4ec0-99a8-28d1444d5f7d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:06.380309 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c38941-d65f-4ec0-99a8-28d1444d5f7d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:06.381859 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c38941-d65f-4ec0-99a8-28d1444d5f7d None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:06.390187 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:06.393940 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-60c38941-d65f-4ec0-99a8-28d1444d5f7d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 25 03:01:06.393940 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:06.393940 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:06.393940 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:06.393940 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:06.393940 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:06.393940 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:06.393940 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:06.393940 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:06.393940 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:06.393940 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:06.393940 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:06.393940 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 25 03:01:06.393940 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 25 03:01:06.393940 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 25 03:01:06.393940 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 25 03:01:06.395107 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:06.395107 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:06.395107 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 25 03:01:06.395107 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 03:01:06.395107 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 25 03:01:06.395107 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 03:01:06.395107 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:06.395107 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:06.395107 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:06.395107 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:06.395107 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:06.395107 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:06.395107 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:06.395107 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:06.395107 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:06.395107 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:06.395107 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:06.395107 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:06.396284 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:06.396284 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:06.396284 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 25 03:01:06.396284 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 25 03:01:06.396284 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 25 03:01:06.396284 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 25 03:01:06.396284 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 25 03:01:06.396284 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 25 03:01:06.396284 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 25 03:01:06.396284 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:06.402334 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 286/571] 10.222.0.58 () {56 vars in 1093 bytes} [Thu Aug 25 03:01:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:06.402334 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c38941-d65f-4ec0-99a8-28d1444d5f7d None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:06.454175 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3127f5aa-dd20-4df9-98a8-43eb4d049408 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:06.492021 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274b1ffb-771d-4676-8e6b-0dbaed38349e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:06.492822 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274b1ffb-771d-4676-8e6b-0dbaed38349e None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:06.493178 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274b1ffb-771d-4676-8e6b-0dbaed38349e None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:06.493454 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274b1ffb-771d-4676-8e6b-0dbaed38349e None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:06.494115 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274b1ffb-771d-4676-8e6b-0dbaed38349e None admin] RBAC: Adding query filter params (name=service) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:06.494481 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274b1ffb-771d-4676-8e6b-0dbaed38349e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:06.495868 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274b1ffb-771d-4676-8e6b-0dbaed38349e None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:06.504089 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 286/572] 10.222.0.58 () {56 vars in 1103 bytes} [Thu Aug 25 03:01:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:06.504874 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274b1ffb-771d-4676-8e6b-0dbaed38349e None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:06.505507 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c38941-d65f-4ec0-99a8-28d1444d5f7d None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:06.541900 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d52c83-969c-445d-8500-54140ac16d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:06.542532 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d52c83-969c-445d-8500-54140ac16d5c None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:06.542810 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d52c83-969c-445d-8500-54140ac16d5c None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:06.543080 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d52c83-969c-445d-8500-54140ac16d5c None admin] PATH_INFO: `/v3/projects` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:06.543766 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d52c83-969c-445d-8500-54140ac16d5c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:06.544068 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d52c83-969c-445d-8500-54140ac16d5c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:06.545575 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d52c83-969c-445d-8500-54140ac16d5c None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:06.557634 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 287/573] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Aug 25 03:01:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:06.558440 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d52c83-969c-445d-8500-54140ac16d5c None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:06.563737 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:06.624642 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274b1ffb-771d-4676-8e6b-0dbaed38349e None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:06.633656 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:06.658820 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d52c83-969c-445d-8500-54140ac16d5c None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:06.659354 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c847164-14eb-4a00-b935-a03a2427ab85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:06.659997 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c847164-14eb-4a00-b935-a03a2427ab85 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:06.660282 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c847164-14eb-4a00-b935-a03a2427ab85 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:06.660556 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c847164-14eb-4a00-b935-a03a2427ab85 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:06.668421 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c847164-14eb-4a00-b935-a03a2427ab85 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:06.669875 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c847164-14eb-4a00-b935-a03a2427ab85 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:06.680774 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-9c847164-14eb-4a00-b935-a03a2427ab85 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 25 03:01:06.680774 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:06.680774 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:06.680774 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:06.680774 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:06.680774 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:06.680774 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:06.680774 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:06.680774 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:06.680774 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:06.680774 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:06.680774 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:06.680774 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 25 03:01:06.680774 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 25 03:01:06.680774 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 25 03:01:06.680774 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 25 03:01:06.681686 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:06.681686 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:06.681686 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 25 03:01:06.681686 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 03:01:06.681686 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 25 03:01:06.681686 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 03:01:06.681686 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:06.681686 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:06.681686 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:06.681686 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:06.681686 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:06.681686 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:06.681686 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:06.681686 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:06.681686 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:06.681686 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:06.681686 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:06.681686 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:06.682653 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:06.682653 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:06.682653 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 25 03:01:06.682653 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 25 03:01:06.682653 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 25 03:01:06.682653 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 25 03:01:06.682653 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 25 03:01:06.682653 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 25 03:01:06.682653 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 25 03:01:06.682653 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:06.682653 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 287/574] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Aug 25 03:01:06 2022] GET /identity/v3/users/glance => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:06.682653 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c847164-14eb-4a00-b935-a03a2427ab85 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:06.694856 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afdc002-35a9-4985-922b-db89dde651e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:06.695467 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afdc002-35a9-4985-922b-db89dde651e1 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:06.695750 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afdc002-35a9-4985-922b-db89dde651e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:06.696023 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afdc002-35a9-4985-922b-db89dde651e1 None admin] PATH_INFO: `/v3/users` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:06.696751 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afdc002-35a9-4985-922b-db89dde651e1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:06.697056 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afdc002-35a9-4985-922b-db89dde651e1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:06.698507 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afdc002-35a9-4985-922b-db89dde651e1 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:06.715955 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 288/575] 10.222.0.58 () {56 vars in 1133 bytes} [Thu Aug 25 03:01:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:06.716680 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afdc002-35a9-4985-922b-db89dde651e1 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:06.774976 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:06.781940 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c847164-14eb-4a00-b935-a03a2427ab85 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:06.814142 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afdc002-35a9-4985-922b-db89dde651e1 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:06.814502 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77947f9c-5e5d-42e0-9526-63529ecf65cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:06.815137 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77947f9c-5e5d-42e0-9526-63529ecf65cb None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:06.815421 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77947f9c-5e5d-42e0-9526-63529ecf65cb None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:06.815697 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77947f9c-5e5d-42e0-9526-63529ecf65cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:06.826505 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77947f9c-5e5d-42e0-9526-63529ecf65cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:06.828673 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77947f9c-5e5d-42e0-9526-63529ecf65cb None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:06.842514 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-77947f9c-5e5d-42e0-9526-63529ecf65cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:06.842514 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:06.842514 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:06.842514 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:06.842514 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:06.842514 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:06.842514 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:06.842514 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:06.842514 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:06.842514 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:06.842514 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:06.842514 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:06.842514 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 03:01:06.842514 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 03:01:06.842514 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 03:01:06.842514 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 03:01:06.844172 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:06.844172 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:06.844172 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:06.844172 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:06.844172 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:06.844172 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:06.844172 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:06.844172 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:06.844172 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:06.844172 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:06.844172 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:06.844172 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:06.844172 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:06.844172 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:06.844172 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:06.844172 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:06.844172 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 03:01:06.845237 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 03:01:06.845237 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:06.845237 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:06.845237 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 288/576] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:01:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:06.845237 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77947f9c-5e5d-42e0-9526-63529ecf65cb None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:06.851451 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e368ce-c8ba-4276-8bec-ecb35898838e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:06.852055 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e368ce-c8ba-4276-8bec-ecb35898838e None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:06.852337 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e368ce-c8ba-4276-8bec-ecb35898838e None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:06.852611 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e368ce-c8ba-4276-8bec-ecb35898838e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:06.853362 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e368ce-c8ba-4276-8bec-ecb35898838e None admin] RBAC: Adding query filter params (role.id=a93ddb1fd6f6459fb46a17573b821422, scope.project.id=9fd4be43faec4e4191e0e4ec4a4588c3, user.id=5047e1a7cd2d416d90b2d8c5a6596522) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:06.853669 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e368ce-c8ba-4276-8bec-ecb35898838e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:06.855146 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e368ce-c8ba-4276-8bec-ecb35898838e None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:06.860436 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 289/577] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Aug 25 03:01:06 2022] GET /identity/v3/role_assignments?user.id=5047e1a7cd2d416d90b2d8c5a6596522&scope.project.id=9fd4be43faec4e4191e0e4ec4a4588c3&role.id=a93ddb1fd6f6459fb46a17573b821422 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:06.861198 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e368ce-c8ba-4276-8bec-ecb35898838e None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:06.877318 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:06.965719 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77947f9c-5e5d-42e0-9526-63529ecf65cb None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:06.982882 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e368ce-c8ba-4276-8bec-ecb35898838e None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:07.003038 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d4c350-cfce-4157-96e6-7ec730a274b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:07.003678 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d4c350-cfce-4157-96e6-7ec730a274b6 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:07.003962 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d4c350-cfce-4157-96e6-7ec730a274b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:07.004233 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d4c350-cfce-4157-96e6-7ec730a274b6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:07.015139 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d4c350-cfce-4157-96e6-7ec730a274b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:01:07.019891 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-f4d4c350-cfce-4157-96e6-7ec730a274b6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 25 03:01:07.019891 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:07.019891 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:07.019891 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:07.019891 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:07.019891 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:07.019891 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:07.019891 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:07.019891 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:07.019891 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:07.019891 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:07.019891 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:07.019891 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:01:07.019891 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:01:07.019891 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:01:07.019891 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:01:07.021001 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:01:07.021001 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:01:07.021001 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:01:07.021001 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:01:07.021001 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:07.021001 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:07.021001 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:07.021001 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:07.021001 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:07.021001 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:07.021001 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:07.021001 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:07.021001 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:07.021001 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:07.021001 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:07.021001 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:07.021846 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:07.021846 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:07.021846 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:07.021846 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:07.021846 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:01:07.021846 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:01:07.021846 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:01:07.021846 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:01:07.021846 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:01:07.021846 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:01:07.021846 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 25 03:01:07.021846 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:07.021846 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 289/578] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Aug 25 03:01:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:07.022557 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d4c350-cfce-4157-96e6-7ec730a274b6 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:07.047030 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:07.056821 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c2bf5b-e59c-4f79-b45c-b00f729ca1a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:07.059215 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c2bf5b-e59c-4f79-b45c-b00f729ca1a3 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:07.059215 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c2bf5b-e59c-4f79-b45c-b00f729ca1a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:07.059215 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c2bf5b-e59c-4f79-b45c-b00f729ca1a3 None admin] PATH_INFO: `/v3/users` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:07.059215 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c2bf5b-e59c-4f79-b45c-b00f729ca1a3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:07.059215 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c2bf5b-e59c-4f79-b45c-b00f729ca1a3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:07.060770 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c2bf5b-e59c-4f79-b45c-b00f729ca1a3 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:07.080901 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 290/579] 10.222.0.58 () {56 vars in 1137 bytes} [Thu Aug 25 03:01:06 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:07.081898 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c2bf5b-e59c-4f79-b45c-b00f729ca1a3 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:07.111887 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:07.130838 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d4c350-cfce-4157-96e6-7ec730a274b6 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:07.167005 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aedb2fa-3b73-4e01-b060-ef57e9eb3858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:07.167642 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aedb2fa-3b73-4e01-b060-ef57e9eb3858 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:07.167921 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aedb2fa-3b73-4e01-b060-ef57e9eb3858 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:07.168197 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aedb2fa-3b73-4e01-b060-ef57e9eb3858 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:07.178739 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aedb2fa-3b73-4e01-b060-ef57e9eb3858 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:07.180177 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aedb2fa-3b73-4e01-b060-ef57e9eb3858 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:07.187295 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c2bf5b-e59c-4f79-b45c-b00f729ca1a3 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:07.195533 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-9aedb2fa-3b73-4e01-b060-ef57e9eb3858 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:07.195533 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:07.195533 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:07.195533 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:07.195533 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:07.195533 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:07.195533 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:07.195533 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:07.195533 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:07.195533 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:07.195533 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:07.195533 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:07.195533 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 03:01:07.195533 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 03:01:07.195533 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 03:01:07.195533 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 03:01:07.196481 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:07.196481 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:07.196481 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:07.196481 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:07.196481 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:07.196481 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:07.196481 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:07.196481 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:07.196481 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:07.196481 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:07.196481 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:07.196481 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:07.196481 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:07.196481 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:07.196481 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:07.196481 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:07.196481 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 03:01:07.197370 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 03:01:07.197370 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:07.197370 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:07.197370 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 290/580] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:01:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:07.197370 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aedb2fa-3b73-4e01-b060-ef57e9eb3858 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:07.228622 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150f50a0-caf9-47fc-b462-6bd69f905fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:07.229229 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150f50a0-caf9-47fc-b462-6bd69f905fcf None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:07.229508 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150f50a0-caf9-47fc-b462-6bd69f905fcf None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:07.229791 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150f50a0-caf9-47fc-b462-6bd69f905fcf None admin] PATH_INFO: `/v3/domains` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:07.230467 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150f50a0-caf9-47fc-b462-6bd69f905fcf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:07.230775 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150f50a0-caf9-47fc-b462-6bd69f905fcf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:07.232125 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150f50a0-caf9-47fc-b462-6bd69f905fcf None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:07.247962 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 291/581] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Aug 25 03:01:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:07.247962 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150f50a0-caf9-47fc-b462-6bd69f905fcf None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:07.299936 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:07.312139 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aedb2fa-3b73-4e01-b060-ef57e9eb3858 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:07.342063 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:07.348820 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94427e4a-670b-4156-a321-61021c40b9bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:07.349495 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94427e4a-670b-4156-a321-61021c40b9bb None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:07.349776 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94427e4a-670b-4156-a321-61021c40b9bb None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:07.350060 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94427e4a-670b-4156-a321-61021c40b9bb None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:07.350761 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94427e4a-670b-4156-a321-61021c40b9bb None admin] RBAC: Adding query filter params (name=service) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:07.351069 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94427e4a-670b-4156-a321-61021c40b9bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:07.352549 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94427e4a-670b-4156-a321-61021c40b9bb None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:07.355946 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150f50a0-caf9-47fc-b462-6bd69f905fcf None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:07.365940 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 291/582] 10.222.0.58 () {56 vars in 1103 bytes} [Thu Aug 25 03:01:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:07.365940 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94427e4a-670b-4156-a321-61021c40b9bb None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:07.398908 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ed3bb4-0d04-409e-b122-9a6fd75cf5fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:07.399700 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ed3bb4-0d04-409e-b122-9a6fd75cf5fc None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:07.399987 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ed3bb4-0d04-409e-b122-9a6fd75cf5fc None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:07.400319 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ed3bb4-0d04-409e-b122-9a6fd75cf5fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:07.411881 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ed3bb4-0d04-409e-b122-9a6fd75cf5fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:07.413464 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ed3bb4-0d04-409e-b122-9a6fd75cf5fc None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:07.428299 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-f9ed3bb4-0d04-409e-b122-9a6fd75cf5fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:07.428299 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:07.428299 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:07.428299 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:07.428299 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:07.428299 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:07.428299 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:07.428299 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:07.428299 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:07.428299 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:07.428299 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:07.428299 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:07.428299 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 03:01:07.428299 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 03:01:07.428299 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 03:01:07.428299 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 03:01:07.429277 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:07.429277 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:07.429277 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:07.429277 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:07.429277 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:07.429277 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:07.429277 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:07.429277 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:07.429277 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:07.429277 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:07.429277 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:07.429277 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:07.429277 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:07.429277 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:07.429277 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:07.429277 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:07.429277 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 03:01:07.430170 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 03:01:07.430170 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:07.430170 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:07.430170 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 292/583] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:01:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:07.430170 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ed3bb4-0d04-409e-b122-9a6fd75cf5fc None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:07.478289 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94427e4a-670b-4156-a321-61021c40b9bb None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:07.513614 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe216483-66ea-4f34-b873-5845b393eaf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:07.514146 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe216483-66ea-4f34-b873-5845b393eaf1 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:07.514967 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe216483-66ea-4f34-b873-5845b393eaf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:07.515243 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe216483-66ea-4f34-b873-5845b393eaf1 None admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:07.515887 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe216483-66ea-4f34-b873-5845b393eaf1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:07.516189 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe216483-66ea-4f34-b873-5845b393eaf1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:07.517548 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe216483-66ea-4f34-b873-5845b393eaf1 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:07.529299 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 292/584] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Aug 25 03:01:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:07.529923 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe216483-66ea-4f34-b873-5845b393eaf1 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:07.542715 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:07.548210 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ed3bb4-0d04-409e-b122-9a6fd75cf5fc None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:07.582411 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275b8b61-f0a2-44ce-b742-6def691d3899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:07.583046 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275b8b61-f0a2-44ce-b742-6def691d3899 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:07.583328 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275b8b61-f0a2-44ce-b742-6def691d3899 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:07.583603 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275b8b61-f0a2-44ce-b742-6def691d3899 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:07.590912 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275b8b61-f0a2-44ce-b742-6def691d3899 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:07.592378 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275b8b61-f0a2-44ce-b742-6def691d3899 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:07.593968 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:07.604742 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-275b8b61-f0a2-44ce-b742-6def691d3899 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 25 03:01:07.604742 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:07.604742 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:07.604742 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:07.604742 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:07.604742 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:07.604742 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:07.604742 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:07.604742 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:07.604742 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:07.604742 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:07.604742 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:07.604742 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 25 03:01:07.604742 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 25 03:01:07.604742 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 25 03:01:07.604742 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 25 03:01:07.609232 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:07.609232 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:07.609232 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 25 03:01:07.609232 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 03:01:07.609232 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 25 03:01:07.609232 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 03:01:07.609232 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:07.609232 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:07.609232 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:07.609232 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:07.609232 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:07.609232 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:07.609232 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:07.609232 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:07.609232 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:07.609232 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:07.609232 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:07.609232 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:07.610188 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:07.610188 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:07.610188 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 25 03:01:07.610188 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 25 03:01:07.610188 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 25 03:01:07.610188 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 25 03:01:07.610188 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 25 03:01:07.610188 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 25 03:01:07.610188 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 25 03:01:07.610188 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:07.610188 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 293/585] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Aug 25 03:01:07 2022] GET /identity/v3/users/cinder => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:07.610188 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275b8b61-f0a2-44ce-b742-6def691d3899 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:07.645954 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe216483-66ea-4f34-b873-5845b393eaf1 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:07.681492 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676bb4ae-d6d5-4176-b0c0-20f651c36978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:07.682123 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676bb4ae-d6d5-4176-b0c0-20f651c36978 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:07.682443 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676bb4ae-d6d5-4176-b0c0-20f651c36978 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:07.682718 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676bb4ae-d6d5-4176-b0c0-20f651c36978 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:07.694039 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676bb4ae-d6d5-4176-b0c0-20f651c36978 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:07.695908 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676bb4ae-d6d5-4176-b0c0-20f651c36978 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:07.711632 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-676bb4ae-d6d5-4176-b0c0-20f651c36978 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:07.711632 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:07.711632 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:07.711632 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:07.711632 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:07.711632 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:07.711632 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:07.711632 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:07.711632 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:07.711632 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:07.711632 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:07.711632 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:07.711632 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 03:01:07.711632 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 03:01:07.711632 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 03:01:07.711632 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 03:01:07.712568 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:07.712568 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:07.712568 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:07.712568 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:07.712568 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:07.712568 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:07.712568 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:07.712568 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:07.712568 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:07.712568 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:07.712568 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:07.712568 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:07.712568 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:07.712568 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:07.712568 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:07.712568 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:07.712568 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 03:01:07.713442 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 03:01:07.713442 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:07.713442 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:07.713442 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 293/586] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:01:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:07.714369 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676bb4ae-d6d5-4176-b0c0-20f651c36978 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:07.720453 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275b8b61-f0a2-44ce-b742-6def691d3899 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:07.754154 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99988e04-d643-430b-a717-959249e97ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:07.754845 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99988e04-d643-430b-a717-959249e97ae4 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:07.755133 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99988e04-d643-430b-a717-959249e97ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:07.755406 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99988e04-d643-430b-a717-959249e97ae4 None admin] PATH_INFO: `/v3/domains` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:07.756048 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99988e04-d643-430b-a717-959249e97ae4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:07.756353 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99988e04-d643-430b-a717-959249e97ae4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:07.757760 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99988e04-d643-430b-a717-959249e97ae4 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:07.771623 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 294/587] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Aug 25 03:01:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:07.772437 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99988e04-d643-430b-a717-959249e97ae4 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:07.792018 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:07.822782 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676bb4ae-d6d5-4176-b0c0-20f651c36978 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:07.833361 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:07.862443 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd466584-d99f-46e7-8d98-db4ec2be1839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:07.863075 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd466584-d99f-46e7-8d98-db4ec2be1839 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:07.863369 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd466584-d99f-46e7-8d98-db4ec2be1839 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:07.863641 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd466584-d99f-46e7-8d98-db4ec2be1839 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:07.870973 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd466584-d99f-46e7-8d98-db4ec2be1839 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:07.872456 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd466584-d99f-46e7-8d98-db4ec2be1839 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:07.884892 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99988e04-d643-430b-a717-959249e97ae4 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:07.885047 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-fd466584-d99f-46e7-8d98-db4ec2be1839 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 03:01:07.885047 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:07.885047 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:07.885047 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:07.885047 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:07.885047 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:07.885047 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:07.885047 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:07.885047 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:07.885047 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:07.885047 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:07.885047 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:07.885047 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 25 03:01:07.885047 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 25 03:01:07.885047 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 25 03:01:07.885873 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 25 03:01:07.885873 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:07.885873 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:07.885873 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:07.885873 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:07.885873 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:07.885873 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:07.885873 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:07.885873 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:07.885873 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:07.885873 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:07.885873 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:07.885873 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:07.885873 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:07.885873 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:07.885873 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:07.885873 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:07.886813 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 25 03:01:07.886813 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 25 03:01:07.886813 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 25 03:01:07.886813 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 25 03:01:07.886813 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 25 03:01:07.886813 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 25 03:01:07.886813 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 03:01:07.886813 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:07.891513 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 294/588] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Aug 25 03:01:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:07.892155 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd466584-d99f-46e7-8d98-db4ec2be1839 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:07.924277 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d45673-ebd8-437b-8222-125865351832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:07.924909 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d45673-ebd8-437b-8222-125865351832 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:07.925195 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d45673-ebd8-437b-8222-125865351832 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:07.925471 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d45673-ebd8-437b-8222-125865351832 None admin] PATH_INFO: `/v3/users` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:07.926228 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d45673-ebd8-437b-8222-125865351832 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:07.926586 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d45673-ebd8-437b-8222-125865351832 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:07.928071 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d45673-ebd8-437b-8222-125865351832 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:07.946893 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 295/589] 10.222.0.58 () {56 vars in 1137 bytes} [Thu Aug 25 03:01:07 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:07.947734 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d45673-ebd8-437b-8222-125865351832 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:08.008425 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd466584-d99f-46e7-8d98-db4ec2be1839 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:08.024915 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:08.045579 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e361297f-c2d0-4bcf-943b-ab4ae8c193fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:08.046488 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e361297f-c2d0-4bcf-943b-ab4ae8c193fd None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:08.046978 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e361297f-c2d0-4bcf-943b-ab4ae8c193fd None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:08.047437 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e361297f-c2d0-4bcf-943b-ab4ae8c193fd None admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:08.048253 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e361297f-c2d0-4bcf-943b-ab4ae8c193fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:08.048824 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e361297f-c2d0-4bcf-943b-ab4ae8c193fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:08.050451 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e361297f-c2d0-4bcf-943b-ab4ae8c193fd None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:08.050561 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d45673-ebd8-437b-8222-125865351832 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:08.063238 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 295/590] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Aug 25 03:01:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:08.064047 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e361297f-c2d0-4bcf-943b-ab4ae8c193fd None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:08.089401 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74ad474-6cf3-4d3f-80b3-35b6212aef63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:08.090304 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74ad474-6cf3-4d3f-80b3-35b6212aef63 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:08.090797 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74ad474-6cf3-4d3f-80b3-35b6212aef63 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:08.091252 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74ad474-6cf3-4d3f-80b3-35b6212aef63 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:08.094814 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:08.099442 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74ad474-6cf3-4d3f-80b3-35b6212aef63 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:08.100888 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74ad474-6cf3-4d3f-80b3-35b6212aef63 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:08.112843 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-f74ad474-6cf3-4d3f-80b3-35b6212aef63 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 03:01:08.112843 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:08.112843 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:08.112843 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:08.112843 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:08.112843 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:08.112843 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:08.112843 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:08.112843 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:08.112843 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:08.112843 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:08.112843 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:08.112843 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 25 03:01:08.112843 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 25 03:01:08.112843 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 25 03:01:08.113926 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 25 03:01:08.113926 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:08.113926 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:08.113926 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:08.113926 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:08.113926 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:08.113926 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:08.113926 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:08.113926 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:08.113926 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:08.113926 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:08.113926 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:08.113926 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:08.113926 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:08.113926 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:08.113926 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:08.113926 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:08.115070 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 25 03:01:08.115070 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 25 03:01:08.115070 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 25 03:01:08.115070 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 25 03:01:08.115070 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 25 03:01:08.115070 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 25 03:01:08.115070 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 03:01:08.115070 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:08.116876 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 296/591] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Aug 25 03:01:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:08.117847 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74ad474-6cf3-4d3f-80b3-35b6212aef63 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:08.179474 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e361297f-c2d0-4bcf-943b-ab4ae8c193fd None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:08.223574 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8934fc-a8bb-4c11-94f8-46e61ece4e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:08.224194 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8934fc-a8bb-4c11-94f8-46e61ece4e68 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:08.224479 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8934fc-a8bb-4c11-94f8-46e61ece4e68 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:08.224757 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8934fc-a8bb-4c11-94f8-46e61ece4e68 None admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:08.225450 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8934fc-a8bb-4c11-94f8-46e61ece4e68 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:08.225756 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8934fc-a8bb-4c11-94f8-46e61ece4e68 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:08.227207 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8934fc-a8bb-4c11-94f8-46e61ece4e68 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:08.231053 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74ad474-6cf3-4d3f-80b3-35b6212aef63 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:08.239703 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 296/592] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Aug 25 03:01:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:08.240534 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8934fc-a8bb-4c11-94f8-46e61ece4e68 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:08.271146 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedbca15-201c-4d2f-816b-95ca53a3025f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:08.272191 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedbca15-201c-4d2f-816b-95ca53a3025f None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:08.272762 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedbca15-201c-4d2f-816b-95ca53a3025f None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:08.273237 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedbca15-201c-4d2f-816b-95ca53a3025f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:08.275667 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:08.285319 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedbca15-201c-4d2f-816b-95ca53a3025f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:08.287149 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedbca15-201c-4d2f-816b-95ca53a3025f None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:08.302687 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-dedbca15-201c-4d2f-816b-95ca53a3025f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:08.302687 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:08.302687 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:08.302687 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:08.302687 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:08.302687 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:08.302687 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:08.302687 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:08.302687 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:08.302687 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:08.302687 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:08.302687 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:08.302687 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 03:01:08.302687 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 03:01:08.302687 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 03:01:08.302687 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 03:01:08.303881 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:08.303881 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:08.303881 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:08.303881 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:08.303881 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:08.303881 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:08.303881 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:08.303881 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:08.303881 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:08.303881 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:08.303881 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:08.303881 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:08.303881 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:08.303881 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:08.303881 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:08.303881 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:08.303881 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 03:01:08.304994 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 03:01:08.304994 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:08.304994 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:08.305653 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 297/593] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:01:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:08.306622 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedbca15-201c-4d2f-816b-95ca53a3025f None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:08.338191 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:08.352617 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8934fc-a8bb-4c11-94f8-46e61ece4e68 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:08.388014 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f448a68f-c72d-4ed0-971a-5c16e5b1fed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:08.388654 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f448a68f-c72d-4ed0-971a-5c16e5b1fed8 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:08.388935 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f448a68f-c72d-4ed0-971a-5c16e5b1fed8 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:08.389207 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f448a68f-c72d-4ed0-971a-5c16e5b1fed8 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:08.396483 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f448a68f-c72d-4ed0-971a-5c16e5b1fed8 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:08.397913 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f448a68f-c72d-4ed0-971a-5c16e5b1fed8 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:08.409233 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-f448a68f-c72d-4ed0-971a-5c16e5b1fed8 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 25 03:01:08.409233 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:08.409233 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:08.409233 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:08.409233 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:08.409233 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:08.409233 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:08.409233 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:08.409233 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:08.409233 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:08.409233 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:08.409233 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:08.409233 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 25 03:01:08.409233 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 25 03:01:08.409233 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 25 03:01:08.409233 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 25 03:01:08.410401 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:08.410401 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:08.410401 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 25 03:01:08.410401 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 03:01:08.410401 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 25 03:01:08.410401 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 03:01:08.410401 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:08.410401 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:08.410401 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:08.410401 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:08.410401 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:08.410401 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:08.410401 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:08.410401 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:08.410401 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:08.410401 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:08.410401 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:08.410401 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:08.411344 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:08.411344 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:08.411344 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 25 03:01:08.411344 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 25 03:01:08.411344 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 25 03:01:08.411344 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 25 03:01:08.411344 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 25 03:01:08.411344 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 25 03:01:08.411344 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 25 03:01:08.411344 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:08.411344 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 297/594] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Aug 25 03:01:08 2022] GET /identity/v3/users/neutron => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:08.411930 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f448a68f-c72d-4ed0-971a-5c16e5b1fed8 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:08.418921 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedbca15-201c-4d2f-816b-95ca53a3025f None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:08.453942 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f59bacc-a420-4531-9577-5ee320fd8f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:08.454828 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f59bacc-a420-4531-9577-5ee320fd8f27 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:08.455279 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f59bacc-a420-4531-9577-5ee320fd8f27 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:08.455697 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f59bacc-a420-4531-9577-5ee320fd8f27 None admin] PATH_INFO: `/v3/projects` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:08.456581 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f59bacc-a420-4531-9577-5ee320fd8f27 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:08.457030 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f59bacc-a420-4531-9577-5ee320fd8f27 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:08.458649 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f59bacc-a420-4531-9577-5ee320fd8f27 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:08.471346 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 298/595] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Aug 25 03:01:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:08.472206 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f59bacc-a420-4531-9577-5ee320fd8f27 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:08.520901 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:08.524180 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f448a68f-c72d-4ed0-971a-5c16e5b1fed8 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:08.560743 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36801082-dcc0-477b-b34d-e56ee41076bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:08.561468 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36801082-dcc0-477b-b34d-e56ee41076bc None admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:08.561773 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36801082-dcc0-477b-b34d-e56ee41076bc None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:08.562132 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36801082-dcc0-477b-b34d-e56ee41076bc None admin] PATH_INFO: `/v3/projects/9fd4be43faec4e4191e0e4ec4a4588c3/users/0cd171bf4df94f8fab397e42835feeab/roles/a93ddb1fd6f6459fb46a17573b821422` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:08.581549 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f59bacc-a420-4531-9577-5ee320fd8f27 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:08.586250 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36801082-dcc0-477b-b34d-e56ee41076bc None admin] RBAC: Authorizing `identity:create_grant(project_id=9fd4be43faec4e4191e0e4ec4a4588c3, user_id=0cd171bf4df94f8fab397e42835feeab, role_id=a93ddb1fd6f6459fb46a17573b821422)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:08.587953 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36801082-dcc0-477b-b34d-e56ee41076bc None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:08.591299 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:08.622747 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada2ecbf-79a0-4ca6-8de0-3d5653e4bd7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:08.623357 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada2ecbf-79a0-4ca6-8de0-3d5653e4bd7a None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:08.623636 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada2ecbf-79a0-4ca6-8de0-3d5653e4bd7a None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:08.623943 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada2ecbf-79a0-4ca6-8de0-3d5653e4bd7a None admin] PATH_INFO: `/v3/domains` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:08.624586 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada2ecbf-79a0-4ca6-8de0-3d5653e4bd7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:08.624888 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada2ecbf-79a0-4ca6-8de0-3d5653e4bd7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:08.626315 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada2ecbf-79a0-4ca6-8de0-3d5653e4bd7a None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:08.638927 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 299/596] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Aug 25 03:01:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:08.639790 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada2ecbf-79a0-4ca6-8de0-3d5653e4bd7a None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:08.651651 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 298/597] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Aug 25 03:01:08 2022] PUT /identity/v3/projects/9fd4be43faec4e4191e0e4ec4a4588c3/users/0cd171bf4df94f8fab397e42835feeab/roles/a93ddb1fd6f6459fb46a17573b821422 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:01:08.653057 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36801082-dcc0-477b-b34d-e56ee41076bc None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:08.790013 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada2ecbf-79a0-4ca6-8de0-3d5653e4bd7a None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:08.801152 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:08.802776 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36801082-dcc0-477b-b34d-e56ee41076bc None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:08.830829 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe0b469-88f9-48af-9e9d-1283887b253f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:08.831365 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe0b469-88f9-48af-9e9d-1283887b253f None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:08.831650 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe0b469-88f9-48af-9e9d-1283887b253f None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:08.831929 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe0b469-88f9-48af-9e9d-1283887b253f None admin] PATH_INFO: `/v3/users` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:08.832673 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe0b469-88f9-48af-9e9d-1283887b253f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:08.832986 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe0b469-88f9-48af-9e9d-1283887b253f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:08.834458 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe0b469-88f9-48af-9e9d-1283887b253f None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:08.847334 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b30d47-13f9-4f72-9797-fcff39e85119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:08.848317 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b30d47-13f9-4f72-9797-fcff39e85119 None admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:08.848776 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b30d47-13f9-4f72-9797-fcff39e85119 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:08.849250 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b30d47-13f9-4f72-9797-fcff39e85119 None admin] PATH_INFO: `/v3/projects/9fd4be43faec4e4191e0e4ec4a4588c3/users/3f89089b390c4b6599e54000c0fa082a/roles/a93ddb1fd6f6459fb46a17573b821422` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:08.854372 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 300/598] 10.222.0.58 () {56 vars in 1139 bytes} [Thu Aug 25 03:01:08 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:08.855247 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe0b469-88f9-48af-9e9d-1283887b253f None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:08.866821 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b30d47-13f9-4f72-9797-fcff39e85119 None admin] RBAC: Authorizing `identity:create_grant(project_id=9fd4be43faec4e4191e0e4ec4a4588c3, user_id=3f89089b390c4b6599e54000c0fa082a, role_id=a93ddb1fd6f6459fb46a17573b821422)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:08.868550 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b30d47-13f9-4f72-9797-fcff39e85119 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:08.925926 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 299/599] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Aug 25 03:01:08 2022] PUT /identity/v3/projects/9fd4be43faec4e4191e0e4ec4a4588c3/users/3f89089b390c4b6599e54000c0fa082a/roles/a93ddb1fd6f6459fb46a17573b821422 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:01:08.927664 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aee1203-19b4-4541-982e-b04937c42be9 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:08.927957 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aee1203-19b4-4541-982e-b04937c42be9 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:08.928234 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aee1203-19b4-4541-982e-b04937c42be9 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:08.929667 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 300/600] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:01:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:01:08.934756 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aee1203-19b4-4541-982e-b04937c42be9 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:08.978115 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe0b469-88f9-48af-9e9d-1283887b253f None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:08.998598 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:09.036382 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c780dad-ac19-4e76-a20c-fddfc570672f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:09.037763 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c780dad-ac19-4e76-a20c-fddfc570672f None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:09.038539 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c780dad-ac19-4e76-a20c-fddfc570672f None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:09.039043 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c780dad-ac19-4e76-a20c-fddfc570672f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:09.049745 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c780dad-ac19-4e76-a20c-fddfc570672f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:09.051968 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c780dad-ac19-4e76-a20c-fddfc570672f None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:09.063569 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aee1203-19b4-4541-982e-b04937c42be9 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:09.070097 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-4c780dad-ac19-4e76-a20c-fddfc570672f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 03:01:09.070097 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:09.070097 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:09.070097 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:09.070097 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:09.070097 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:09.070097 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:09.070097 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:09.070097 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:09.070097 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:09.070097 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:09.070097 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:09.070097 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 25 03:01:09.070097 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 25 03:01:09.070097 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 25 03:01:09.071052 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 25 03:01:09.071052 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:09.071052 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:09.071052 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:09.071052 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:09.071052 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:09.071052 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:09.071052 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:09.071052 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:09.071052 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:09.071052 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:09.071052 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:09.071052 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:09.071052 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:09.071052 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:09.071052 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:09.071052 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:09.071940 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 25 03:01:09.071940 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 25 03:01:09.071940 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 25 03:01:09.071940 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 25 03:01:09.071940 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 25 03:01:09.071940 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 25 03:01:09.071940 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 03:01:09.071940 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:09.074683 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 301/601] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Aug 25 03:01:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:09.076102 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71163efa-d236-4207-b760-a5696041b4bd None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:09.076587 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71163efa-d236-4207-b760-a5696041b4bd None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:09.077079 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71163efa-d236-4207-b760-a5696041b4bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:09.098841 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-71163efa-d236-4207-b760-a5696041b4bd None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:01:09.100327 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-71163efa-d236-4207-b760-a5696041b4bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:01:09.106035 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b3696e-6a22-4a1b-a67a-777c44579ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:09.106744 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b3696e-6a22-4a1b-a67a-777c44579ab4 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:09.107022 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b3696e-6a22-4a1b-a67a-777c44579ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:09.107368 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b3696e-6a22-4a1b-a67a-777c44579ab4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:09.120183 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:09.125001 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b3696e-6a22-4a1b-a67a-777c44579ab4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:09.128245 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b3696e-6a22-4a1b-a67a-777c44579ab4 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:09.149196 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-88b3696e-6a22-4a1b-a67a-777c44579ab4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:09.149196 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:09.149196 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:09.149196 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:09.149196 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:09.149196 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:09.149196 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:09.149196 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:09.149196 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:09.149196 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:09.149196 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:09.149196 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:09.149196 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 03:01:09.149196 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 03:01:09.149196 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 03:01:09.149196 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 03:01:09.150332 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:09.150332 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:09.150332 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:09.150332 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:09.150332 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:09.150332 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:09.150332 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:09.150332 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:09.150332 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:09.150332 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:09.150332 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:09.150332 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:09.150332 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:09.150332 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:09.150332 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:09.150332 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:09.150332 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 03:01:09.151222 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 03:01:09.151222 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:09.151222 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:09.152157 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 301/602] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:01:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:09.154065 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b3696e-6a22-4a1b-a67a-777c44579ab4 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:09.183853 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 302/603] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:01:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 03:01:09.185188 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71163efa-d236-4207-b760-a5696041b4bd None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:09.293803 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71163efa-d236-4207-b760-a5696041b4bd None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:09.305874 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:09.310697 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b3696e-6a22-4a1b-a67a-777c44579ab4 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:09.333141 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1c4540-13d3-4a5b-b7d3-d9021a5202c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:09.334143 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1c4540-13d3-4a5b-b7d3-d9021a5202c5 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:09.334664 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1c4540-13d3-4a5b-b7d3-d9021a5202c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:09.335180 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1c4540-13d3-4a5b-b7d3-d9021a5202c5 None admin] PATH_INFO: `/v3/domains` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:09.336136 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1c4540-13d3-4a5b-b7d3-d9021a5202c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:09.336641 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1c4540-13d3-4a5b-b7d3-d9021a5202c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:09.338428 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1c4540-13d3-4a5b-b7d3-d9021a5202c5 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:09.351554 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 303/604] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Aug 25 03:01:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:09.356544 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1c4540-13d3-4a5b-b7d3-d9021a5202c5 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:09.377361 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0d9cb0-d8bf-4775-91dc-cc08bf9d80d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:09.378311 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0d9cb0-d8bf-4775-91dc-cc08bf9d80d4 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:09.378838 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0d9cb0-d8bf-4775-91dc-cc08bf9d80d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:09.380803 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0d9cb0-d8bf-4775-91dc-cc08bf9d80d4 None admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:09.381718 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0d9cb0-d8bf-4775-91dc-cc08bf9d80d4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:09.382228 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0d9cb0-d8bf-4775-91dc-cc08bf9d80d4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:09.385396 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0d9cb0-d8bf-4775-91dc-cc08bf9d80d4 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:09.395475 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:09.412949 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 302/605] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Aug 25 03:01:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:09.418868 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0d9cb0-d8bf-4775-91dc-cc08bf9d80d4 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:09.528288 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1c4540-13d3-4a5b-b7d3-d9021a5202c5 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:09.567934 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed79138-e93d-4fb0-a9a9-92780d860ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:09.569100 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed79138-e93d-4fb0-a9a9-92780d860ec7 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:09.569535 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed79138-e93d-4fb0-a9a9-92780d860ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:09.570018 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed79138-e93d-4fb0-a9a9-92780d860ec7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:09.580389 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed79138-e93d-4fb0-a9a9-92780d860ec7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:09.582977 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed79138-e93d-4fb0-a9a9-92780d860ec7 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:09.597690 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-1ed79138-e93d-4fb0-a9a9-92780d860ec7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 03:01:09.597690 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:09.597690 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:09.597690 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:09.597690 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:09.597690 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:09.597690 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:09.597690 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:09.597690 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:09.597690 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:09.597690 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:09.597690 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:09.597690 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 25 03:01:09.597690 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 25 03:01:09.597690 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 25 03:01:09.598871 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 25 03:01:09.598871 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:09.598871 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:09.598871 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:09.598871 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:09.598871 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:09.598871 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:09.598871 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:09.598871 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:09.598871 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:09.598871 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:09.598871 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:09.598871 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:09.598871 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:09.598871 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:09.598871 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:09.598871 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:09.599975 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 25 03:01:09.599975 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 25 03:01:09.599975 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 25 03:01:09.599975 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 25 03:01:09.599975 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 25 03:01:09.599975 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 25 03:01:09.599975 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 03:01:09.599975 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:09.602071 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 304/606] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Aug 25 03:01:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:09.603145 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed79138-e93d-4fb0-a9a9-92780d860ec7 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:09.616940 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0d9cb0-d8bf-4775-91dc-cc08bf9d80d4 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:09.660925 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886b6d5b-3dd3-472d-aae6-0fb60d26e613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:09.661626 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886b6d5b-3dd3-472d-aae6-0fb60d26e613 None admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:09.661860 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886b6d5b-3dd3-472d-aae6-0fb60d26e613 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:09.662127 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886b6d5b-3dd3-472d-aae6-0fb60d26e613 None admin] PATH_INFO: `/v3/projects/9fd4be43faec4e4191e0e4ec4a4588c3/users/60193ba9e1c14df3bc9eebcc6f28f2d7/roles/a93ddb1fd6f6459fb46a17573b821422` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:09.683378 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886b6d5b-3dd3-472d-aae6-0fb60d26e613 None admin] RBAC: Authorizing `identity:create_grant(project_id=9fd4be43faec4e4191e0e4ec4a4588c3, user_id=60193ba9e1c14df3bc9eebcc6f28f2d7, role_id=a93ddb1fd6f6459fb46a17573b821422)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:09.684835 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886b6d5b-3dd3-472d-aae6-0fb60d26e613 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:09.689042 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:09.714094 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed79138-e93d-4fb0-a9a9-92780d860ec7 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:09.749563 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 303/607] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Aug 25 03:01:09 2022] PUT /identity/v3/projects/9fd4be43faec4e4191e0e4ec4a4588c3/users/60193ba9e1c14df3bc9eebcc6f28f2d7/roles/a93ddb1fd6f6459fb46a17573b821422 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:01:09.751278 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886b6d5b-3dd3-472d-aae6-0fb60d26e613 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:09.759458 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468f28f9-42ac-4800-889d-aabf10fe6edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:09.762398 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468f28f9-42ac-4800-889d-aabf10fe6edd None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:09.762677 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468f28f9-42ac-4800-889d-aabf10fe6edd None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:09.763972 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468f28f9-42ac-4800-889d-aabf10fe6edd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:09.775814 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:09.792440 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468f28f9-42ac-4800-889d-aabf10fe6edd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:01:09.804129 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-468f28f9-42ac-4800-889d-aabf10fe6edd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 25 03:01:09.804129 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:09.804129 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:09.804129 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:09.804129 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:09.804129 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:09.804129 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:09.804129 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:09.804129 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:09.804129 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:09.804129 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:09.804129 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:09.804129 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:01:09.804129 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:01:09.804129 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:01:09.804129 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:01:09.805592 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:01:09.805592 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:01:09.805592 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:01:09.805592 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:01:09.805592 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:09.805592 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:09.805592 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:09.805592 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:09.805592 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:09.805592 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:09.805592 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:09.805592 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:09.805592 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:09.805592 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:09.805592 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:09.805592 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:09.806508 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:09.806508 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:09.806508 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:09.806508 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:09.806508 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:01:09.806508 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:01:09.806508 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:01:09.806508 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:01:09.806508 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:01:09.806508 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:01:09.806508 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 25 03:01:09.806508 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:09.808931 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 305/608] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Aug 25 03:01:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:09.815142 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468f28f9-42ac-4800-889d-aabf10fe6edd None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:09.936296 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886b6d5b-3dd3-472d-aae6-0fb60d26e613 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:09.976758 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3ff63d-bf0d-4d60-937e-644c38b0af04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:09.977432 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3ff63d-bf0d-4d60-937e-644c38b0af04 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:09.977722 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3ff63d-bf0d-4d60-937e-644c38b0af04 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:09.978055 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3ff63d-bf0d-4d60-937e-644c38b0af04 None admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:09.978852 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3ff63d-bf0d-4d60-937e-644c38b0af04 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:09.979161 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3ff63d-bf0d-4d60-937e-644c38b0af04 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:09.980034 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468f28f9-42ac-4800-889d-aabf10fe6edd None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:09.980596 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3ff63d-bf0d-4d60-937e-644c38b0af04 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:09.997511 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 304/609] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Aug 25 03:01:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:10.003979 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3ff63d-bf0d-4d60-937e-644c38b0af04 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:10.025912 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06afce50-1217-4c1d-ba4a-b8eee3543b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:10.027172 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06afce50-1217-4c1d-ba4a-b8eee3543b24 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:10.028024 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06afce50-1217-4c1d-ba4a-b8eee3543b24 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:10.028553 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06afce50-1217-4c1d-ba4a-b8eee3543b24 None admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:10.029514 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06afce50-1217-4c1d-ba4a-b8eee3543b24 None admin] RBAC: Adding query filter params (name=service) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:10.030662 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06afce50-1217-4c1d-ba4a-b8eee3543b24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:10.033089 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06afce50-1217-4c1d-ba4a-b8eee3543b24 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:10.035707 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:10.049617 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 306/610] 10.222.0.58 () {56 vars in 1103 bytes} [Thu Aug 25 03:01:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:10.054950 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06afce50-1217-4c1d-ba4a-b8eee3543b24 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:10.100204 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:10.171597 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3ff63d-bf0d-4d60-937e-644c38b0af04 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:10.214678 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06afce50-1217-4c1d-ba4a-b8eee3543b24 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:10.238999 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fdb38d-90b2-4033-b914-3fcc35d87f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:10.239676 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fdb38d-90b2-4033-b914-3fcc35d87f55 None admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:10.239959 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fdb38d-90b2-4033-b914-3fcc35d87f55 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:10.240238 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fdb38d-90b2-4033-b914-3fcc35d87f55 None admin] PATH_INFO: `/v3/projects/9fd4be43faec4e4191e0e4ec4a4588c3/users/a0a1a7f091774842907fd08de15e8058/roles/a93ddb1fd6f6459fb46a17573b821422` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:10.271682 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fdb38d-90b2-4033-b914-3fcc35d87f55 None admin] RBAC: Authorizing `identity:create_grant(project_id=9fd4be43faec4e4191e0e4ec4a4588c3, user_id=a0a1a7f091774842907fd08de15e8058, role_id=a93ddb1fd6f6459fb46a17573b821422)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:10.273142 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fdb38d-90b2-4033-b914-3fcc35d87f55 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:10.282046 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dca968-e334-46a1-ade4-1a0a63679657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:10.282712 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dca968-e334-46a1-ade4-1a0a63679657 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:10.282992 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dca968-e334-46a1-ade4-1a0a63679657 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:10.283270 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dca968-e334-46a1-ade4-1a0a63679657 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:10.294704 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dca968-e334-46a1-ade4-1a0a63679657 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:10.303352 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dca968-e334-46a1-ade4-1a0a63679657 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:10.311862 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-38dca968-e334-46a1-ade4-1a0a63679657 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:10.311862 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:10.311862 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:10.311862 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:10.311862 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:10.311862 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:10.311862 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:10.311862 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:10.311862 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:10.311862 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:10.311862 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:10.311862 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:10.311862 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 03:01:10.311862 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 03:01:10.311862 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 03:01:10.311862 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 03:01:10.313103 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:10.313103 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:10.313103 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:10.313103 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:10.313103 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:10.313103 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:10.313103 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:10.313103 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:10.313103 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:10.313103 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:10.313103 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:10.313103 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:10.313103 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:10.313103 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:10.313103 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:10.313103 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:10.313103 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 03:01:10.314223 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 03:01:10.314223 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:10.314223 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:10.317880 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 307/611] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:01:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:10.319075 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dca968-e334-46a1-ade4-1a0a63679657 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:10.366037 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 305/612] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Aug 25 03:01:10 2022] PUT /identity/v3/projects/9fd4be43faec4e4191e0e4ec4a4588c3/users/a0a1a7f091774842907fd08de15e8058/roles/a93ddb1fd6f6459fb46a17573b821422 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:01:10.382517 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:10.435319 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dca968-e334-46a1-ade4-1a0a63679657 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:10.472646 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6274f2cf-eb9d-4aa2-a8f3-9d8c53beef57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:10.473192 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6274f2cf-eb9d-4aa2-a8f3-9d8c53beef57 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:10.473533 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6274f2cf-eb9d-4aa2-a8f3-9d8c53beef57 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:10.473814 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6274f2cf-eb9d-4aa2-a8f3-9d8c53beef57 None admin] PATH_INFO: `/v3/domains` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:10.474506 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6274f2cf-eb9d-4aa2-a8f3-9d8c53beef57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:10.474813 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6274f2cf-eb9d-4aa2-a8f3-9d8c53beef57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:10.476209 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6274f2cf-eb9d-4aa2-a8f3-9d8c53beef57 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:10.493616 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 308/613] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Aug 25 03:01:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:01:10.495143 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fdb38d-90b2-4033-b914-3fcc35d87f55 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:10.611094 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fdb38d-90b2-4033-b914-3fcc35d87f55 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:10.632141 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:10.654115 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae217a3-9afc-405c-9923-1d9dfbaa0ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:10.654786 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae217a3-9afc-405c-9923-1d9dfbaa0ced None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:10.655695 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae217a3-9afc-405c-9923-1d9dfbaa0ced None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:10.655986 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae217a3-9afc-405c-9923-1d9dfbaa0ced None admin] PATH_INFO: `/v3/users/swift` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:10.663463 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae217a3-9afc-405c-9923-1d9dfbaa0ced None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:10.664979 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae217a3-9afc-405c-9923-1d9dfbaa0ced None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:10.679606 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-eae217a3-9afc-405c-9923-1d9dfbaa0ced None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 25 03:01:10.679606 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:10.679606 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:10.679606 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:10.679606 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:10.679606 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:10.679606 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:10.679606 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:10.679606 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:10.679606 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:10.679606 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:10.679606 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:10.679606 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 25 03:01:10.679606 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 25 03:01:10.679606 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 25 03:01:10.679606 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 25 03:01:10.680554 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:10.680554 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:10.680554 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 25 03:01:10.680554 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 03:01:10.680554 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 25 03:01:10.680554 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 03:01:10.680554 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:10.680554 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:10.680554 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:10.680554 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:10.680554 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:10.680554 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:10.680554 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:10.680554 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:10.680554 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:10.680554 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:10.680554 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:10.680554 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:10.681470 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:10.681470 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:10.681470 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 25 03:01:10.681470 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 25 03:01:10.681470 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 25 03:01:10.681470 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 25 03:01:10.681470 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 25 03:01:10.681470 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 25 03:01:10.681470 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 25 03:01:10.681470 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:10.682572 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 306/614] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Aug 25 03:01:10 2022] GET /identity/v3/users/swift => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:10.686995 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6274f2cf-eb9d-4aa2-a8f3-9d8c53beef57 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:10.783864 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693e3845-e324-4460-b0c5-246692192df6 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:10.784063 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693e3845-e324-4460-b0c5-246692192df6 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:10.784530 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693e3845-e324-4460-b0c5-246692192df6 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:10.790345 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 307/615] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:01:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:01:10.792166 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b32aee-71f8-44d6-aa2e-cffd0c3722a7 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:10.792610 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b32aee-71f8-44d6-aa2e-cffd0c3722a7 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:10.792899 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b32aee-71f8-44d6-aa2e-cffd0c3722a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:10.809951 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6274f2cf-eb9d-4aa2-a8f3-9d8c53beef57 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:10.813044 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-81b32aee-71f8-44d6-aa2e-cffd0c3722a7 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:01:10.814304 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-81b32aee-71f8-44d6-aa2e-cffd0c3722a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:01:10.826818 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:10.891682 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2c1e30-2d14-4556-8fbf-76a75f6a68e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:10.892674 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2c1e30-2d14-4556-8fbf-76a75f6a68e1 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:10.893179 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2c1e30-2d14-4556-8fbf-76a75f6a68e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:10.893653 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2c1e30-2d14-4556-8fbf-76a75f6a68e1 None admin] PATH_INFO: `/v3/users` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:10.894722 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2c1e30-2d14-4556-8fbf-76a75f6a68e1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:10.895258 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2c1e30-2d14-4556-8fbf-76a75f6a68e1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:10.896905 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2c1e30-2d14-4556-8fbf-76a75f6a68e1 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:10.904544 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 308/616] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:01:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 25 03:01:10.921495 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 309/617] 10.222.0.58 () {56 vars in 1135 bytes} [Thu Aug 25 03:01:10 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:10.926821 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b32aee-71f8-44d6-aa2e-cffd0c3722a7 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:11.063293 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b32aee-71f8-44d6-aa2e-cffd0c3722a7 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:11.073048 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:11.107132 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2facc4a1-1d0c-4c0f-8b88-25cdf64d4b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:11.107780 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2facc4a1-1d0c-4c0f-8b88-25cdf64d4b91 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:11.108104 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2facc4a1-1d0c-4c0f-8b88-25cdf64d4b91 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:11.108918 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2facc4a1-1d0c-4c0f-8b88-25cdf64d4b91 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:11.125368 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2facc4a1-1d0c-4c0f-8b88-25cdf64d4b91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:11.127054 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2facc4a1-1d0c-4c0f-8b88-25cdf64d4b91 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:11.143829 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-2facc4a1-1d0c-4c0f-8b88-25cdf64d4b91 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:11.143829 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:11.143829 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:11.143829 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:11.143829 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:11.143829 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:11.143829 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:11.143829 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:11.143829 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:11.143829 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:11.143829 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:11.143829 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:11.143829 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 03:01:11.143829 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 03:01:11.143829 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 03:01:11.143829 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 03:01:11.144810 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:11.144810 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:11.144810 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:11.144810 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:11.144810 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:11.144810 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:11.144810 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:11.144810 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:11.144810 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:11.144810 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:11.144810 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:11.144810 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:11.144810 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:11.144810 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:11.144810 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:11.144810 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:11.144810 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 03:01:11.145682 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 03:01:11.145682 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:11.145682 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:11.148521 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 309/618] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:01:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:11.151197 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2c1e30-2d14-4556-8fbf-76a75f6a68e1 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:11.249842 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3c8e48-8e77-4ed3-b7b5-dd7c60f26f0c None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:11.250119 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3c8e48-8e77-4ed3-b7b5-dd7c60f26f0c None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:11.250419 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3c8e48-8e77-4ed3-b7b5-dd7c60f26f0c None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:11.252021 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 310/619] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:01:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 25 03:01:11.258088 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e4a468-df2c-445a-9daa-6ae6ac87fb2d None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:11.258406 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e4a468-df2c-445a-9daa-6ae6ac87fb2d None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:11.258683 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e4a468-df2c-445a-9daa-6ae6ac87fb2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:11.259541 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2c1e30-2d14-4556-8fbf-76a75f6a68e1 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:11.277382 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-80e4a468-df2c-445a-9daa-6ae6ac87fb2d None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:01:11.278578 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-80e4a468-df2c-445a-9daa-6ae6ac87fb2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:01:11.300263 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91abc7f-cf49-4f52-a5c4-296ae86e2b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:11.301178 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91abc7f-cf49-4f52-a5c4-296ae86e2b1e None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:11.301703 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91abc7f-cf49-4f52-a5c4-296ae86e2b1e None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:11.302262 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91abc7f-cf49-4f52-a5c4-296ae86e2b1e None admin] PATH_INFO: `/v3/domains` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:11.303236 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91abc7f-cf49-4f52-a5c4-296ae86e2b1e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:11.303777 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91abc7f-cf49-4f52-a5c4-296ae86e2b1e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:11.305464 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91abc7f-cf49-4f52-a5c4-296ae86e2b1e None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:11.319644 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 310/620] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Aug 25 03:01:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:11.320738 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91abc7f-cf49-4f52-a5c4-296ae86e2b1e None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:11.347284 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 311/621] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:01:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 25 03:01:11.348137 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e4a468-df2c-445a-9daa-6ae6ac87fb2d None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:11.352511 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:11.462670 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:11.466612 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91abc7f-cf49-4f52-a5c4-296ae86e2b1e None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:11.470997 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e4a468-df2c-445a-9daa-6ae6ac87fb2d None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:11.506672 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd6aae4-1b90-40a8-aa53-8d83ee653858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:11.507310 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd6aae4-1b90-40a8-aa53-8d83ee653858 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:11.507593 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd6aae4-1b90-40a8-aa53-8d83ee653858 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:11.507867 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd6aae4-1b90-40a8-aa53-8d83ee653858 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:11.522307 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd6aae4-1b90-40a8-aa53-8d83ee653858 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:11.522307 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd6aae4-1b90-40a8-aa53-8d83ee653858 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:11.527814 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-6dd6aae4-1b90-40a8-aa53-8d83ee653858 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 03:01:11.527814 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:11.527814 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:11.527814 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:11.527814 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:11.527814 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:11.527814 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:11.527814 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:11.527814 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:11.527814 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:11.527814 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:11.527814 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:11.527814 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 25 03:01:11.527814 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 25 03:01:11.527814 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 25 03:01:11.528716 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 25 03:01:11.528716 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:11.528716 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:11.528716 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:11.528716 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:11.528716 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:11.528716 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:11.528716 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:11.528716 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:11.528716 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:11.528716 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:11.528716 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:11.528716 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:11.528716 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:11.528716 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:11.528716 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:11.528716 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:11.529596 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 25 03:01:11.529596 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 25 03:01:11.529596 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 25 03:01:11.529596 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 25 03:01:11.529596 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 25 03:01:11.529596 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 25 03:01:11.529596 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 03:01:11.529596 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:11.529596 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 312/622] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Aug 25 03:01:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:11.535392 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd6aae4-1b90-40a8-aa53-8d83ee653858 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:11.559678 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e799b053-45fb-4548-bfe1-934f74f962d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:11.560359 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e799b053-45fb-4548-bfe1-934f74f962d2 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:11.560648 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e799b053-45fb-4548-bfe1-934f74f962d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:11.560925 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e799b053-45fb-4548-bfe1-934f74f962d2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:11.614964 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e799b053-45fb-4548-bfe1-934f74f962d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:01:11.619666 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-e799b053-45fb-4548-bfe1-934f74f962d2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 25 03:01:11.619666 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:11.619666 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:11.619666 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:11.619666 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:11.619666 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:11.619666 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:11.619666 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:11.619666 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:11.619666 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:11.619666 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:11.619666 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:11.619666 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:01:11.619666 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:01:11.619666 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:01:11.619666 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:01:11.620578 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:01:11.620578 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:01:11.620578 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:01:11.620578 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:01:11.620578 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:11.620578 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:11.620578 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:11.620578 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:11.620578 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:11.620578 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:11.620578 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:11.620578 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:11.620578 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:11.620578 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:11.620578 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:11.620578 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:11.621414 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:11.621414 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:11.621414 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:11.621414 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:11.621414 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:01:11.621414 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:01:11.621414 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:01:11.621414 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:01:11.621414 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:01:11.621414 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:01:11.621414 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 25 03:01:11.621414 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:11.621414 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 311/623] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Aug 25 03:01:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:11.629671 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e799b053-45fb-4548-bfe1-934f74f962d2 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:11.645921 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd6aae4-1b90-40a8-aa53-8d83ee653858 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:11.683845 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfa5737-314f-4ae9-9230-82e4ad89c968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:11.684929 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfa5737-314f-4ae9-9230-82e4ad89c968 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:11.685193 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfa5737-314f-4ae9-9230-82e4ad89c968 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:11.686861 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfa5737-314f-4ae9-9230-82e4ad89c968 None admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:11.687128 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfa5737-314f-4ae9-9230-82e4ad89c968 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:11.687369 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfa5737-314f-4ae9-9230-82e4ad89c968 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:11.688667 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfa5737-314f-4ae9-9230-82e4ad89c968 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:11.702945 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 313/624] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Aug 25 03:01:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:11.711336 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfa5737-314f-4ae9-9230-82e4ad89c968 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:11.733861 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:11.786707 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:11.803227 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e799b053-45fb-4548-bfe1-934f74f962d2 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:11.841374 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21b8cf0-a108-4af9-bfa9-8665b8aa419e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:11.842030 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21b8cf0-a108-4af9-bfa9-8665b8aa419e None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:11.842300 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21b8cf0-a108-4af9-bfa9-8665b8aa419e None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:11.842680 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21b8cf0-a108-4af9-bfa9-8665b8aa419e None admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:11.843351 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21b8cf0-a108-4af9-bfa9-8665b8aa419e None admin] RBAC: Adding query filter params (name=service) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:11.843706 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21b8cf0-a108-4af9-bfa9-8665b8aa419e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:11.845062 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfa5737-314f-4ae9-9230-82e4ad89c968 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:11.845187 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21b8cf0-a108-4af9-bfa9-8665b8aa419e None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:11.859128 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 312/625] 10.222.0.58 () {56 vars in 1103 bytes} [Thu Aug 25 03:01:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:11.859128 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21b8cf0-a108-4af9-bfa9-8665b8aa419e None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:11.895383 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092220f4-e3f4-4964-8fe8-90db087ef1ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:11.896153 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092220f4-e3f4-4964-8fe8-90db087ef1ae None admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:11.896541 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092220f4-e3f4-4964-8fe8-90db087ef1ae None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:11.896943 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092220f4-e3f4-4964-8fe8-90db087ef1ae None admin] PATH_INFO: `/v3/projects/9fd4be43faec4e4191e0e4ec4a4588c3/users/5047e1a7cd2d416d90b2d8c5a6596522/roles/a93ddb1fd6f6459fb46a17573b821422` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:11.918737 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092220f4-e3f4-4964-8fe8-90db087ef1ae None admin] RBAC: Authorizing `identity:create_grant(project_id=9fd4be43faec4e4191e0e4ec4a4588c3, user_id=5047e1a7cd2d416d90b2d8c5a6596522, role_id=a93ddb1fd6f6459fb46a17573b821422)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:11.921418 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092220f4-e3f4-4964-8fe8-90db087ef1ae None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:11.978541 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21b8cf0-a108-4af9-bfa9-8665b8aa419e None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:11.979848 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 314/626] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Aug 25 03:01:11 2022] PUT /identity/v3/projects/9fd4be43faec4e4191e0e4ec4a4588c3/users/5047e1a7cd2d416d90b2d8c5a6596522/roles/a93ddb1fd6f6459fb46a17573b821422 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:01:11.981151 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092220f4-e3f4-4964-8fe8-90db087ef1ae None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:12.060016 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ac6741-f7cf-447d-b8c9-30d26a97fab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:12.062227 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ac6741-f7cf-447d-b8c9-30d26a97fab0 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:12.062557 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ac6741-f7cf-447d-b8c9-30d26a97fab0 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:12.062831 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ac6741-f7cf-447d-b8c9-30d26a97fab0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:12.087283 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ac6741-f7cf-447d-b8c9-30d26a97fab0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:01:12.094728 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-96ac6741-f7cf-447d-b8c9-30d26a97fab0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 25 03:01:12.094728 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:12.094728 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:12.094728 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:12.094728 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:12.094728 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:12.094728 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:12.094728 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:12.094728 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:12.094728 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:12.094728 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:12.094728 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:12.094728 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:01:12.094728 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:01:12.094728 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:01:12.094728 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:01:12.095737 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:01:12.095737 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:01:12.095737 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:01:12.095737 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:01:12.095737 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:12.095737 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:12.095737 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:12.095737 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:12.095737 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:12.095737 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:12.095737 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:12.095737 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:12.095737 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:12.095737 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:12.095737 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:12.095737 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:12.096592 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:12.096592 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:12.096592 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:12.096592 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:12.096592 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:01:12.096592 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:01:12.096592 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:01:12.096592 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:01:12.096592 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:01:12.096592 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:01:12.096592 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 25 03:01:12.096592 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:12.096592 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 313/627] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Aug 25 03:01:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:12.097849 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90610b7-9475-442b-a19d-b737fecab186 None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:12.097849 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90610b7-9475-442b-a19d-b737fecab186 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:12.098046 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90610b7-9475-442b-a19d-b737fecab186 None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:12.104634 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 314/628] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:01:12 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:01:12.109297 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90610b7-9475-442b-a19d-b737fecab186 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:12.117119 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:12.118383 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:12.164062 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092220f4-e3f4-4964-8fe8-90db087ef1ae None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:12.201403 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e826041c-e13a-4689-a4d7-ffdc04f27a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:12.202933 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e826041c-e13a-4689-a4d7-ffdc04f27a12 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:12.203306 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e826041c-e13a-4689-a4d7-ffdc04f27a12 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:12.203654 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e826041c-e13a-4689-a4d7-ffdc04f27a12 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:12.215712 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e826041c-e13a-4689-a4d7-ffdc04f27a12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:12.217359 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e826041c-e13a-4689-a4d7-ffdc04f27a12 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:12.218592 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90610b7-9475-442b-a19d-b737fecab186 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:12.234525 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-e826041c-e13a-4689-a4d7-ffdc04f27a12 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:12.234525 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:12.234525 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:12.234525 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:12.234525 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:12.234525 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:12.234525 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:12.234525 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:12.234525 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:12.234525 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:12.234525 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:12.234525 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:12.234525 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 03:01:12.234525 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 03:01:12.234525 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 03:01:12.234525 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 03:01:12.235605 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:12.235605 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:12.235605 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:12.235605 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:12.235605 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:12.235605 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:12.235605 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:12.235605 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:12.235605 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:12.235605 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:12.235605 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:12.235605 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:12.235605 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:12.235605 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:12.235605 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:12.235605 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:12.235605 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 03:01:12.236470 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 03:01:12.236470 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:12.236470 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:12.236470 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 315/629] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:01:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:12.242102 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfb6abb-f056-4284-94a9-a47950434753 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:12.242871 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfb6abb-f056-4284-94a9-a47950434753 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:12.242871 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfb6abb-f056-4284-94a9-a47950434753 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:12.257945 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6233d2-cac1-42e9-8009-ebb154f2ff38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:12.258857 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6233d2-cac1-42e9-8009-ebb154f2ff38 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:12.259322 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6233d2-cac1-42e9-8009-ebb154f2ff38 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:12.259776 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6233d2-cac1-42e9-8009-ebb154f2ff38 None admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:12.260626 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6233d2-cac1-42e9-8009-ebb154f2ff38 None admin] RBAC: Adding query filter params (name=service) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:12.261133 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6233d2-cac1-42e9-8009-ebb154f2ff38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:12.262464 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-9cfb6abb-f056-4284-94a9-a47950434753 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:01:12.262958 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6233d2-cac1-42e9-8009-ebb154f2ff38 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:12.263791 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-9cfb6abb-f056-4284-94a9-a47950434753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:01:12.272963 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 315/630] 10.222.0.58 () {56 vars in 1103 bytes} [Thu Aug 25 03:01:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:12.278329 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6233d2-cac1-42e9-8009-ebb154f2ff38 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:12.344411 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 316/631] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:01:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 03:01:12.344411 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfb6abb-f056-4284-94a9-a47950434753 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:12.365632 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:12.386218 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:12.419556 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6233d2-cac1-42e9-8009-ebb154f2ff38 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:12.447989 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfb6abb-f056-4284-94a9-a47950434753 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:12.461904 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940c7747-235a-464e-bb28-104e872bea0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:12.462635 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c7747-235a-464e-bb28-104e872bea0a None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:12.462923 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c7747-235a-464e-bb28-104e872bea0a None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:12.463231 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c7747-235a-464e-bb28-104e872bea0a None admin] PATH_INFO: `/v3/domains` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:12.463891 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940c7747-235a-464e-bb28-104e872bea0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:12.464188 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940c7747-235a-464e-bb28-104e872bea0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:12.465641 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940c7747-235a-464e-bb28-104e872bea0a None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:12.479075 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 316/632] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Aug 25 03:01:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:12.491361 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940c7747-235a-464e-bb28-104e872bea0a None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:12.498101 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0601fc1f-cfb2-455f-8b21-57af5121fe4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:12.499100 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0601fc1f-cfb2-455f-8b21-57af5121fe4a None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:12.499588 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0601fc1f-cfb2-455f-8b21-57af5121fe4a None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:12.500060 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0601fc1f-cfb2-455f-8b21-57af5121fe4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:12.511138 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0601fc1f-cfb2-455f-8b21-57af5121fe4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:12.512898 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0601fc1f-cfb2-455f-8b21-57af5121fe4a None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:12.528501 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-0601fc1f-cfb2-455f-8b21-57af5121fe4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:12.528501 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:12.528501 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:12.528501 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:12.528501 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:12.528501 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:12.528501 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:12.528501 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:12.528501 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:12.528501 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:12.528501 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:12.528501 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:12.528501 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 03:01:12.528501 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 03:01:12.528501 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 03:01:12.528501 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 03:01:12.529460 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:12.529460 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:12.529460 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:12.529460 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:12.529460 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:12.529460 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:12.529460 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:12.529460 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:12.529460 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:12.529460 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:12.529460 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:12.529460 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:12.529460 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:12.529460 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:12.529460 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:12.529460 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:12.529460 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 03:01:12.530900 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 03:01:12.530900 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:12.530900 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:12.532966 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 317/633] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:01:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:12.538788 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0601fc1f-cfb2-455f-8b21-57af5121fe4a None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:12.643831 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:12.651571 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0601fc1f-cfb2-455f-8b21-57af5121fe4a None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:12.681156 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940c7747-235a-464e-bb28-104e872bea0a None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:12.694509 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f61759-0fc6-48fa-906c-95f50b761172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:12.695771 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f61759-0fc6-48fa-906c-95f50b761172 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:12.696266 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f61759-0fc6-48fa-906c-95f50b761172 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:12.696754 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f61759-0fc6-48fa-906c-95f50b761172 None admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:12.697610 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f61759-0fc6-48fa-906c-95f50b761172 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:12.698202 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f61759-0fc6-48fa-906c-95f50b761172 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:12.699823 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f61759-0fc6-48fa-906c-95f50b761172 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:12.730923 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 318/634] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Aug 25 03:01:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:01:12.734925 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f61759-0fc6-48fa-906c-95f50b761172 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:12.754635 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:12.762689 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e0fd25-8848-47df-b8c1-0abd9d621e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:12.766229 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e0fd25-8848-47df-b8c1-0abd9d621e34 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:12.766229 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e0fd25-8848-47df-b8c1-0abd9d621e34 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:12.766229 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e0fd25-8848-47df-b8c1-0abd9d621e34 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:12.778119 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e0fd25-8848-47df-b8c1-0abd9d621e34 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:12.779610 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e0fd25-8848-47df-b8c1-0abd9d621e34 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:12.797339 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-58e0fd25-8848-47df-b8c1-0abd9d621e34 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 25 03:01:12.797339 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:12.797339 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:12.797339 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:12.797339 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:12.797339 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:12.797339 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:12.797339 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:12.797339 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:12.797339 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:12.797339 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:12.797339 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:12.797339 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 25 03:01:12.797339 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 25 03:01:12.797339 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 25 03:01:12.797339 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 25 03:01:12.798356 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:12.798356 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:12.798356 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 25 03:01:12.798356 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 03:01:12.798356 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 25 03:01:12.798356 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 03:01:12.798356 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:12.798356 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:12.798356 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:12.798356 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:12.798356 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:12.798356 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:12.798356 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:12.798356 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:12.798356 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:12.798356 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:12.798356 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:12.798356 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:12.799291 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:12.799291 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:12.799291 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 25 03:01:12.799291 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 25 03:01:12.799291 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 25 03:01:12.799291 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 25 03:01:12.799291 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 25 03:01:12.799291 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 25 03:01:12.799291 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 25 03:01:12.799291 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:12.799291 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 317/635] 10.222.0.58 () {56 vars in 1093 bytes} [Thu Aug 25 03:01:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:12.800365 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31b265a-5661-44f9-80ce-8fef187de834 None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:12.800365 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31b265a-5661-44f9-80ce-8fef187de834 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:12.800528 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31b265a-5661-44f9-80ce-8fef187de834 None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:12.805654 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 318/636] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:01:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:01:12.807405 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31b265a-5661-44f9-80ce-8fef187de834 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:12.867687 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f61759-0fc6-48fa-906c-95f50b761172 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:12.901568 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaec6612-534d-4166-b468-0f6e240d31eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:12.902202 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaec6612-534d-4166-b468-0f6e240d31eb None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:12.902527 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaec6612-534d-4166-b468-0f6e240d31eb None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:12.902803 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaec6612-534d-4166-b468-0f6e240d31eb None admin] PATH_INFO: `/v3/users/glance` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:12.910573 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31b265a-5661-44f9-80ce-8fef187de834 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:12.911058 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaec6612-534d-4166-b468-0f6e240d31eb None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:12.912040 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaec6612-534d-4166-b468-0f6e240d31eb None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:12.923864 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-eaec6612-534d-4166-b468-0f6e240d31eb None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 25 03:01:12.923864 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:12.923864 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:12.923864 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:12.923864 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:12.923864 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:12.923864 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:12.923864 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:12.923864 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:12.923864 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:12.923864 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:12.923864 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:12.923864 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 25 03:01:12.923864 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 25 03:01:12.923864 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 25 03:01:12.923864 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 25 03:01:12.924806 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:12.924806 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:12.924806 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 25 03:01:12.924806 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 03:01:12.924806 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 25 03:01:12.924806 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 03:01:12.924806 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:12.924806 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:12.924806 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:12.924806 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:12.924806 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:12.924806 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:12.924806 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:12.924806 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:12.924806 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:12.924806 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:12.924806 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:12.924806 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:12.925758 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:12.925758 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:12.925758 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 25 03:01:12.925758 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 25 03:01:12.925758 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 25 03:01:12.925758 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 25 03:01:12.925758 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 25 03:01:12.925758 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 25 03:01:12.925758 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 25 03:01:12.925758 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:12.925758 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 319/637] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Aug 25 03:01:12 2022] GET /identity/v3/users/glance => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:12.926389 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb9a137-95ed-4279-98a4-ac397f734522 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:12.926389 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb9a137-95ed-4279-98a4-ac397f734522 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:12.927054 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb9a137-95ed-4279-98a4-ac397f734522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:12.944767 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-adb9a137-95ed-4279-98a4-ac397f734522 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:01:12.945873 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-adb9a137-95ed-4279-98a4-ac397f734522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:01:12.951088 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf9ad41-ab0d-497e-a2f5-1a12b627e8fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:12.951788 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf9ad41-ab0d-497e-a2f5-1a12b627e8fb None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:12.952096 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf9ad41-ab0d-497e-a2f5-1a12b627e8fb None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:12.952367 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf9ad41-ab0d-497e-a2f5-1a12b627e8fb None admin] PATH_INFO: `/v3/users` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:12.953350 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf9ad41-ab0d-497e-a2f5-1a12b627e8fb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:12.953711 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf9ad41-ab0d-497e-a2f5-1a12b627e8fb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:12.955094 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf9ad41-ab0d-497e-a2f5-1a12b627e8fb None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:12.956065 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:12.973302 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 319/638] 10.222.0.58 () {56 vars in 1133 bytes} [Thu Aug 25 03:01:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:12.974046 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf9ad41-ab0d-497e-a2f5-1a12b627e8fb None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:13.023855 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 320/639] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:01:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 03:01:13.025086 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb9a137-95ed-4279-98a4-ac397f734522 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:13.034381 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:13.089600 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf9ad41-ab0d-497e-a2f5-1a12b627e8fb None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:13.127991 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb68e0c-c375-44f3-a0b4-5580bd6b883f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:13.128894 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb68e0c-c375-44f3-a0b4-5580bd6b883f None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:13.129323 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb68e0c-c375-44f3-a0b4-5580bd6b883f None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:13.129735 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb68e0c-c375-44f3-a0b4-5580bd6b883f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:13.131105 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb9a137-95ed-4279-98a4-ac397f734522 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:13.141022 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb68e0c-c375-44f3-a0b4-5580bd6b883f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:01:13.147374 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-afb68e0c-c375-44f3-a0b4-5580bd6b883f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 25 03:01:13.147374 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:13.147374 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:13.147374 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:13.147374 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:13.147374 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:13.147374 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:13.147374 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:13.147374 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:13.147374 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:13.147374 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:13.147374 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:13.147374 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:01:13.147374 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:01:13.147374 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:01:13.147374 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:01:13.148453 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:01:13.148453 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:01:13.148453 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:01:13.148453 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:01:13.148453 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:13.148453 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:13.148453 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:13.148453 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:13.148453 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:13.148453 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:13.148453 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:13.148453 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:13.148453 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:13.148453 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:13.148453 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:13.148453 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:13.149492 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:13.149492 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:13.149492 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:13.149492 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:13.149492 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:01:13.149492 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:01:13.149492 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:01:13.149492 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:01:13.149492 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:01:13.149492 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:01:13.149492 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 25 03:01:13.149492 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:13.151980 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 320/640] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Aug 25 03:01:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:13.153024 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb68e0c-c375-44f3-a0b4-5580bd6b883f None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:13.175299 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe35224a-b3d7-4ec2-bcbb-6716902992dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:13.176049 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe35224a-b3d7-4ec2-bcbb-6716902992dd None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:13.176449 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe35224a-b3d7-4ec2-bcbb-6716902992dd None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:13.176822 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe35224a-b3d7-4ec2-bcbb-6716902992dd None admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:13.177769 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe35224a-b3d7-4ec2-bcbb-6716902992dd None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:13.178177 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe35224a-b3d7-4ec2-bcbb-6716902992dd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:13.179720 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe35224a-b3d7-4ec2-bcbb-6716902992dd None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:13.198040 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 321/641] 10.222.0.58 () {56 vars in 1137 bytes} [Thu Aug 25 03:01:13 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:13.203667 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe35224a-b3d7-4ec2-bcbb-6716902992dd None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:13.206536 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:13.260161 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb68e0c-c375-44f3-a0b4-5580bd6b883f None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:13.269798 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:13.295625 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091a7b5c-7902-4dcc-8008-59f4276e6b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:13.296763 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091a7b5c-7902-4dcc-8008-59f4276e6b01 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:13.297040 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091a7b5c-7902-4dcc-8008-59f4276e6b01 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:13.297313 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091a7b5c-7902-4dcc-8008-59f4276e6b01 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:13.315489 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe35224a-b3d7-4ec2-bcbb-6716902992dd None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:13.352510 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c31ff70-a782-4706-bc98-602469f3a4ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:13.353237 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c31ff70-a782-4706-bc98-602469f3a4ef None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:13.353519 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c31ff70-a782-4706-bc98-602469f3a4ef None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:13.353906 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c31ff70-a782-4706-bc98-602469f3a4ef None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:13.354609 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c31ff70-a782-4706-bc98-602469f3a4ef None admin] RBAC: Adding query filter params (name=service) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:13.354916 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c31ff70-a782-4706-bc98-602469f3a4ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:13.356426 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c31ff70-a782-4706-bc98-602469f3a4ef None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:13.365916 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 322/642] 10.222.0.58 () {56 vars in 1103 bytes} [Thu Aug 25 03:01:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:13.366946 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091a7b5c-7902-4dcc-8008-59f4276e6b01 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:13.368383 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091a7b5c-7902-4dcc-8008-59f4276e6b01 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:13.371403 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c31ff70-a782-4706-bc98-602469f3a4ef None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:13.435802 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:13.471417 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 321/643] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Aug 25 03:01:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 25 03:01:13.472531 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091a7b5c-7902-4dcc-8008-59f4276e6b01 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:13.482525 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:13.484624 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c31ff70-a782-4706-bc98-602469f3a4ef None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:13.523180 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3a0b6c-e1ca-4455-89ce-4a61f294ca33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:13.523908 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3a0b6c-e1ca-4455-89ce-4a61f294ca33 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:13.524191 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3a0b6c-e1ca-4455-89ce-4a61f294ca33 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:13.524578 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3a0b6c-e1ca-4455-89ce-4a61f294ca33 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:13.573072 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091a7b5c-7902-4dcc-8008-59f4276e6b01 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:13.602687 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3a0b6c-e1ca-4455-89ce-4a61f294ca33 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:13.604293 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3a0b6c-e1ca-4455-89ce-4a61f294ca33 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:13.607743 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7352a8e0-31d3-4577-a757-0f9eaab14f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:13.608428 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7352a8e0-31d3-4577-a757-0f9eaab14f33 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:13.608711 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7352a8e0-31d3-4577-a757-0f9eaab14f33 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:13.608987 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7352a8e0-31d3-4577-a757-0f9eaab14f33 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:13.620155 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7352a8e0-31d3-4577-a757-0f9eaab14f33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:01:13.624885 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-7352a8e0-31d3-4577-a757-0f9eaab14f33 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 25 03:01:13.624885 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:13.624885 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:13.624885 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:13.624885 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:13.624885 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:13.624885 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:13.624885 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:13.624885 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:13.624885 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:13.624885 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:13.624885 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:13.624885 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:01:13.624885 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:01:13.624885 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:01:13.624885 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:01:13.625893 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:01:13.625893 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:01:13.625893 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:01:13.625893 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:01:13.625893 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:13.625893 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:13.625893 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:13.625893 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:13.625893 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:13.625893 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:13.625893 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:13.625893 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:13.625893 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:13.625893 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:13.625893 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:13.625893 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:13.632283 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:13.632283 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:13.632283 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:13.632283 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:13.632283 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:01:13.632283 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:01:13.632283 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:01:13.632283 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:01:13.632283 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:01:13.632283 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:01:13.632283 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 25 03:01:13.632283 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:13.632283 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 322/644] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Aug 25 03:01:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:13.632283 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7352a8e0-31d3-4577-a757-0f9eaab14f33 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:13.672981 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:13.711367 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:13.714182 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 323/645] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Aug 25 03:01:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 25 03:01:13.715067 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3a0b6c-e1ca-4455-89ce-4a61f294ca33 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:13.759962 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7352a8e0-31d3-4577-a757-0f9eaab14f33 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:13.810698 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0acf264-5ccc-4fbb-be51-fd7181d6f42c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:13.811846 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0acf264-5ccc-4fbb-be51-fd7181d6f42c None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:13.812136 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0acf264-5ccc-4fbb-be51-fd7181d6f42c None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:13.812412 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0acf264-5ccc-4fbb-be51-fd7181d6f42c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:13.821723 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3a0b6c-e1ca-4455-89ce-4a61f294ca33 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:13.823231 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0acf264-5ccc-4fbb-be51-fd7181d6f42c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:13.824704 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0acf264-5ccc-4fbb-be51-fd7181d6f42c None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:13.839549 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-f0acf264-5ccc-4fbb-be51-fd7181d6f42c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:13.839549 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:13.839549 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:13.839549 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:13.839549 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:13.839549 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:13.839549 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:13.839549 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:13.839549 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:13.839549 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:13.839549 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:13.839549 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:13.839549 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 03:01:13.839549 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 03:01:13.839549 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 03:01:13.839549 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 03:01:13.840502 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:13.840502 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:13.840502 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:13.840502 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:13.840502 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:13.840502 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:13.840502 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:13.840502 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:13.840502 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:13.840502 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:13.840502 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:13.840502 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:13.840502 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:13.840502 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:13.840502 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:13.840502 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:13.840502 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 03:01:13.841573 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 03:01:13.841573 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:13.841573 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:13.841573 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 323/646] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:01:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:13.841573 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0acf264-5ccc-4fbb-be51-fd7181d6f42c None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:13.863890 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6e9a90-57b8-4a3d-af0f-722770b0954f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:13.864834 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6e9a90-57b8-4a3d-af0f-722770b0954f None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:13.865298 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6e9a90-57b8-4a3d-af0f-722770b0954f None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:13.865719 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6e9a90-57b8-4a3d-af0f-722770b0954f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:13.877093 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6e9a90-57b8-4a3d-af0f-722770b0954f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:13.878729 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6e9a90-57b8-4a3d-af0f-722770b0954f None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:13.894985 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-3b6e9a90-57b8-4a3d-af0f-722770b0954f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:13.894985 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:13.894985 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:13.894985 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:13.894985 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:13.894985 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:13.894985 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:13.894985 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:13.894985 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:13.894985 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:13.894985 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:13.894985 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:13.894985 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 03:01:13.894985 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 03:01:13.894985 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 03:01:13.894985 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 03:01:13.896168 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:13.896168 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:13.896168 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:13.896168 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:13.896168 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:13.896168 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:13.896168 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:13.896168 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:13.896168 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:13.896168 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:13.896168 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:13.896168 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:13.896168 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:13.896168 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:13.896168 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:13.896168 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:13.896168 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 03:01:13.897318 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 03:01:13.897318 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:13.897318 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:13.898730 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 324/647] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:01:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:13.899642 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6e9a90-57b8-4a3d-af0f-722770b0954f None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:13.922659 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:13.954978 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0acf264-5ccc-4fbb-be51-fd7181d6f42c None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:13.974334 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:13.988878 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3f2330-bbb2-4b4c-84a6-37ecbd78abd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:13.989481 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3f2330-bbb2-4b4c-84a6-37ecbd78abd7 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:13.989762 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3f2330-bbb2-4b4c-84a6-37ecbd78abd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:13.990088 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3f2330-bbb2-4b4c-84a6-37ecbd78abd7 None admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:13.990800 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3f2330-bbb2-4b4c-84a6-37ecbd78abd7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:13.991112 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3f2330-bbb2-4b4c-84a6-37ecbd78abd7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:13.992498 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3f2330-bbb2-4b4c-84a6-37ecbd78abd7 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:14.000882 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 324/648] 10.222.0.58 () {56 vars in 1103 bytes} [Thu Aug 25 03:01:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:14.001661 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3f2330-bbb2-4b4c-84a6-37ecbd78abd7 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:14.006315 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6e9a90-57b8-4a3d-af0f-722770b0954f None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:14.040475 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50fc86f-9e7a-4317-8208-a7b980231e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:14.041031 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50fc86f-9e7a-4317-8208-a7b980231e43 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:14.041309 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50fc86f-9e7a-4317-8208-a7b980231e43 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:14.041584 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50fc86f-9e7a-4317-8208-a7b980231e43 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:14.052354 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50fc86f-9e7a-4317-8208-a7b980231e43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:14.053777 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50fc86f-9e7a-4317-8208-a7b980231e43 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:14.067348 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-d50fc86f-9e7a-4317-8208-a7b980231e43 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:14.067348 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:14.067348 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:14.067348 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:14.067348 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:14.067348 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:14.067348 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:14.067348 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:14.067348 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:14.067348 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:14.067348 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:14.067348 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:14.067348 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 03:01:14.067348 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 03:01:14.067348 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 03:01:14.067348 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 03:01:14.068521 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:14.068521 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:14.068521 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:14.068521 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:14.068521 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:14.068521 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:14.068521 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:14.068521 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:14.068521 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:14.068521 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:14.068521 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:14.068521 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:14.068521 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:14.068521 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:14.068521 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:14.068521 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:14.068521 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 03:01:14.069603 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 03:01:14.069603 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:14.069603 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:14.069603 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 325/649] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:01:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:14.069603 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50fc86f-9e7a-4317-8208-a7b980231e43 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:14.101116 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3f2330-bbb2-4b4c-84a6-37ecbd78abd7 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:14.133136 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:14.134010 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c68ce2-2ebc-49d6-aa37-7e2c354ce2d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:14.134638 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c68ce2-2ebc-49d6-aa37-7e2c354ce2d2 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:14.134921 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c68ce2-2ebc-49d6-aa37-7e2c354ce2d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:14.135191 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c68ce2-2ebc-49d6-aa37-7e2c354ce2d2 None admin] PATH_INFO: `/v3/domains` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:14.135818 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c68ce2-2ebc-49d6-aa37-7e2c354ce2d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:14.136116 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c68ce2-2ebc-49d6-aa37-7e2c354ce2d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:14.137490 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c68ce2-2ebc-49d6-aa37-7e2c354ce2d2 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:14.149028 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 325/650] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Aug 25 03:01:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:14.150209 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b8e00a-a9cc-4544-b8f4-16ed01399fec None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:14.150522 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b8e00a-a9cc-4544-b8f4-16ed01399fec None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:14.150797 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b8e00a-a9cc-4544-b8f4-16ed01399fec None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:14.152179 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 326/651] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:01:14 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:01:14.152651 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b8e00a-a9cc-4544-b8f4-16ed01399fec None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:14.170159 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50fc86f-9e7a-4317-8208-a7b980231e43 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:14.199154 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:14.203420 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbbd851-6b51-4d8c-995c-141c350aaa2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:14.204027 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbbd851-6b51-4d8c-995c-141c350aaa2d None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:14.204306 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbbd851-6b51-4d8c-995c-141c350aaa2d None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:14.204578 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbbd851-6b51-4d8c-995c-141c350aaa2d None admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:14.205222 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbbd851-6b51-4d8c-995c-141c350aaa2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:14.205526 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbbd851-6b51-4d8c-995c-141c350aaa2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:14.206929 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbbd851-6b51-4d8c-995c-141c350aaa2d None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:14.219282 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 326/652] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Aug 25 03:01:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:14.220197 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbbd851-6b51-4d8c-995c-141c350aaa2d None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:14.259774 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b8e00a-a9cc-4544-b8f4-16ed01399fec None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:14.296092 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917b54ae-a2f9-4a87-81a8-71fa26653fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:14.296746 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917b54ae-a2f9-4a87-81a8-71fa26653fef None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:14.297024 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917b54ae-a2f9-4a87-81a8-71fa26653fef None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:14.297297 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917b54ae-a2f9-4a87-81a8-71fa26653fef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:14.307050 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917b54ae-a2f9-4a87-81a8-71fa26653fef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:14.308519 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917b54ae-a2f9-4a87-81a8-71fa26653fef None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:14.322455 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-917b54ae-a2f9-4a87-81a8-71fa26653fef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:14.322455 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:14.322455 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:14.322455 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:14.322455 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:14.322455 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:14.322455 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:14.322455 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:14.322455 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:14.322455 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:14.322455 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:14.322455 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:14.322455 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 03:01:14.322455 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 03:01:14.322455 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 03:01:14.322455 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 03:01:14.323666 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:14.323666 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:14.323666 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:14.323666 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:14.323666 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:14.323666 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:14.323666 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:14.323666 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:14.323666 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:14.323666 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:14.323666 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:14.323666 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:14.323666 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:14.323666 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:14.323666 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:14.323666 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:14.323666 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 03:01:14.324863 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 03:01:14.324863 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:14.324863 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:14.324863 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 327/653] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:01:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:14.324863 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917b54ae-a2f9-4a87-81a8-71fa26653fef None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:14.326633 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbbd851-6b51-4d8c-995c-141c350aaa2d None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:14.354781 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:14.362696 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faa1eab-e10d-40c3-a220-5ee3dc74ca83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:14.367895 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faa1eab-e10d-40c3-a220-5ee3dc74ca83 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:14.367895 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faa1eab-e10d-40c3-a220-5ee3dc74ca83 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:14.367895 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faa1eab-e10d-40c3-a220-5ee3dc74ca83 None admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:14.367895 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faa1eab-e10d-40c3-a220-5ee3dc74ca83 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:14.367895 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faa1eab-e10d-40c3-a220-5ee3dc74ca83 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:14.367895 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faa1eab-e10d-40c3-a220-5ee3dc74ca83 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:14.377379 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 327/654] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Aug 25 03:01:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:14.378685 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068df140-a2be-445b-a991-c9779123eda1 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:14.378978 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068df140-a2be-445b-a991-c9779123eda1 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:14.379252 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068df140-a2be-445b-a991-c9779123eda1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:14.396415 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-068df140-a2be-445b-a991-c9779123eda1 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:01:14.397594 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-068df140-a2be-445b-a991-c9779123eda1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:01:14.422406 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:14.429190 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917b54ae-a2f9-4a87-81a8-71fa26653fef None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:14.462414 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6f38f3-65e7-4fdf-b92b-2ff8e0d67133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:14.463037 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6f38f3-65e7-4fdf-b92b-2ff8e0d67133 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:14.463318 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6f38f3-65e7-4fdf-b92b-2ff8e0d67133 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:14.463588 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6f38f3-65e7-4fdf-b92b-2ff8e0d67133 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:14.463873 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 328/655] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:01:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 03:01:14.464698 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068df140-a2be-445b-a991-c9779123eda1 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:14.488571 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6f38f3-65e7-4fdf-b92b-2ff8e0d67133 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:14.490357 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6f38f3-65e7-4fdf-b92b-2ff8e0d67133 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:14.503394 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-fd6f38f3-65e7-4fdf-b92b-2ff8e0d67133 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 25 03:01:14.503394 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:14.503394 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:14.503394 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:14.503394 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:14.503394 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:14.503394 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:14.503394 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:14.503394 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:14.503394 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:14.503394 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:14.503394 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:14.503394 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 25 03:01:14.503394 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 25 03:01:14.503394 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 25 03:01:14.503394 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 25 03:01:14.504548 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:14.504548 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:14.504548 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 25 03:01:14.504548 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 03:01:14.504548 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 25 03:01:14.504548 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 03:01:14.504548 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:14.504548 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:14.504548 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:14.504548 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:14.504548 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:14.504548 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:14.504548 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:14.504548 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:14.504548 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:14.504548 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:14.504548 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:14.504548 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:14.505621 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:14.505621 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:14.505621 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 25 03:01:14.505621 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 25 03:01:14.505621 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 25 03:01:14.505621 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 25 03:01:14.505621 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 25 03:01:14.505621 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 25 03:01:14.505621 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 25 03:01:14.505621 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:14.505621 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 328/656] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Aug 25 03:01:14 2022] GET /identity/v3/users/cinder => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:14.506520 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6f38f3-65e7-4fdf-b92b-2ff8e0d67133 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:14.569498 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068df140-a2be-445b-a991-c9779123eda1 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:14.593492 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:14.613682 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f381327-d428-4b3a-a5ee-ff1b0ff79885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:14.613873 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6f38f3-65e7-4fdf-b92b-2ff8e0d67133 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:14.614568 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f381327-d428-4b3a-a5ee-ff1b0ff79885 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:14.614941 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f381327-d428-4b3a-a5ee-ff1b0ff79885 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:14.615568 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f381327-d428-4b3a-a5ee-ff1b0ff79885 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:14.625333 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f381327-d428-4b3a-a5ee-ff1b0ff79885 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:14.627338 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f381327-d428-4b3a-a5ee-ff1b0ff79885 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:14.648063 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-5f381327-d428-4b3a-a5ee-ff1b0ff79885 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 03:01:14.648063 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:14.648063 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:14.648063 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:14.648063 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:14.648063 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:14.648063 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:14.648063 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:14.648063 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:14.648063 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:14.648063 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:14.648063 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:14.648063 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 25 03:01:14.648063 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 25 03:01:14.648063 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 25 03:01:14.649144 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 25 03:01:14.649144 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:14.649144 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:14.649144 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:14.649144 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:14.649144 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:14.649144 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:14.649144 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:14.649144 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:14.649144 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:14.649144 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:14.649144 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:14.649144 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:14.649144 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:14.649144 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:14.649144 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:14.649144 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:14.650385 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 25 03:01:14.650385 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 25 03:01:14.650385 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 25 03:01:14.650385 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 25 03:01:14.650385 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 25 03:01:14.650385 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 25 03:01:14.650385 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 03:01:14.650385 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:14.650385 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 329/657] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Aug 25 03:01:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:14.650385 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f381327-d428-4b3a-a5ee-ff1b0ff79885 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:14.654916 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc3e6fa-206b-447d-b59b-f22397bb10f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:14.655568 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc3e6fa-206b-447d-b59b-f22397bb10f2 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:14.655852 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc3e6fa-206b-447d-b59b-f22397bb10f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:14.656127 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc3e6fa-206b-447d-b59b-f22397bb10f2 None admin] PATH_INFO: `/v3/domains` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:14.656773 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc3e6fa-206b-447d-b59b-f22397bb10f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:14.657076 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc3e6fa-206b-447d-b59b-f22397bb10f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:14.658478 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc3e6fa-206b-447d-b59b-f22397bb10f2 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:14.670497 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 329/658] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Aug 25 03:01:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:14.671319 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc3e6fa-206b-447d-b59b-f22397bb10f2 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:14.687370 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:14.766300 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f381327-d428-4b3a-a5ee-ff1b0ff79885 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:14.771618 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc3e6fa-206b-447d-b59b-f22397bb10f2 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:14.808115 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0739d4-c287-4b51-88c5-604abfca5666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:14.808751 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0739d4-c287-4b51-88c5-604abfca5666 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:14.809029 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0739d4-c287-4b51-88c5-604abfca5666 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:14.809155 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0638bbf-d138-411a-b73f-5da542e8b167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:14.809302 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0739d4-c287-4b51-88c5-604abfca5666 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:14.809757 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0638bbf-d138-411a-b73f-5da542e8b167 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:14.810039 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0638bbf-d138-411a-b73f-5da542e8b167 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:14.810342 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0638bbf-d138-411a-b73f-5da542e8b167 None admin] PATH_INFO: `/v3/users` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:14.811102 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0638bbf-d138-411a-b73f-5da542e8b167 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:14.811927 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0638bbf-d138-411a-b73f-5da542e8b167 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:14.813354 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0638bbf-d138-411a-b73f-5da542e8b167 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:14.818577 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0739d4-c287-4b51-88c5-604abfca5666 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:14.820046 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0739d4-c287-4b51-88c5-604abfca5666 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:14.830932 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 330/659] 10.222.0.58 () {56 vars in 1137 bytes} [Thu Aug 25 03:01:14 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:14.831738 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0638bbf-d138-411a-b73f-5da542e8b167 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:14.832770 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-af0739d4-c287-4b51-88c5-604abfca5666 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 03:01:14.832770 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:14.832770 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:14.832770 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:14.832770 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:14.832770 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:14.832770 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:14.832770 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:14.832770 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:14.832770 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:14.832770 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:14.832770 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:14.832770 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 25 03:01:14.832770 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 25 03:01:14.832770 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 25 03:01:14.835004 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 25 03:01:14.835004 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:14.835004 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:14.835004 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:14.835004 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:14.835004 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:14.835004 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:14.835004 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:14.835004 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:14.835004 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:14.835004 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:14.835004 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:14.835004 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:14.835004 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:14.835004 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:14.835004 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:14.835004 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:14.836904 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 25 03:01:14.836904 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 25 03:01:14.836904 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 25 03:01:14.836904 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 25 03:01:14.836904 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 25 03:01:14.836904 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 25 03:01:14.836904 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 03:01:14.836904 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:14.836904 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 330/660] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Aug 25 03:01:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:14.836904 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0739d4-c287-4b51-88c5-604abfca5666 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:14.876611 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:14.908694 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:14.933837 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0739d4-c287-4b51-88c5-604abfca5666 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:14.938186 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0638bbf-d138-411a-b73f-5da542e8b167 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:14.968664 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7a5641-de80-4378-9215-c755f74e974a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:14.969289 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7a5641-de80-4378-9215-c755f74e974a None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:14.969567 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7a5641-de80-4378-9215-c755f74e974a None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:14.969842 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7a5641-de80-4378-9215-c755f74e974a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:14.972399 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50223a98-3e58-4989-90a3-33a67ecd029e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:14.973006 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50223a98-3e58-4989-90a3-33a67ecd029e None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:14.973289 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50223a98-3e58-4989-90a3-33a67ecd029e None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:14.973565 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50223a98-3e58-4989-90a3-33a67ecd029e None admin] PATH_INFO: `/v3/projects` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:14.974257 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50223a98-3e58-4989-90a3-33a67ecd029e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:14.974596 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50223a98-3e58-4989-90a3-33a67ecd029e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:14.976123 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50223a98-3e58-4989-90a3-33a67ecd029e None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:14.976639 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7a5641-de80-4378-9215-c755f74e974a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:14.978067 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7a5641-de80-4378-9215-c755f74e974a None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:14.988876 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 331/661] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Aug 25 03:01:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:14.989627 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-ca7a5641-de80-4378-9215-c755f74e974a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 25 03:01:14.989627 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:14.989627 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:14.989627 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:14.989627 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:14.989627 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:14.989627 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:14.989627 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:14.989627 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:14.989627 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:14.989627 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:14.989627 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:14.989627 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 25 03:01:14.989627 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 25 03:01:14.989627 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 25 03:01:14.989627 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 25 03:01:14.990624 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:14.990624 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:14.990624 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 25 03:01:14.990624 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 03:01:14.990624 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 25 03:01:14.990624 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 03:01:14.990624 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:14.990624 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:14.990624 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:14.990624 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:14.990624 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:14.990624 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:14.990624 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:14.990624 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:14.990624 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:14.990624 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:14.990624 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:14.990624 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:14.991585 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:14.991585 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:14.991585 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 25 03:01:14.991585 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 25 03:01:14.991585 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 25 03:01:14.991585 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 25 03:01:14.991585 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 25 03:01:14.991585 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 25 03:01:14.991585 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 25 03:01:14.991585 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:14.992211 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50223a98-3e58-4989-90a3-33a67ecd029e None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:14.992297 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 331/662] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Aug 25 03:01:14 2022] GET /identity/v3/users/neutron => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:14.992297 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7a5641-de80-4378-9215-c755f74e974a None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:15.090229 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:15.094211 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7a5641-de80-4378-9215-c755f74e974a None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:15.094505 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50223a98-3e58-4989-90a3-33a67ecd029e None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:15.127041 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:15.131732 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab79860-ba78-44a9-adfe-9b5d71867b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:15.132007 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3371fa1b-871a-4fbc-a5df-c299d4464808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:15.132640 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3371fa1b-871a-4fbc-a5df-c299d4464808 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:15.132921 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3371fa1b-871a-4fbc-a5df-c299d4464808 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:15.133197 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3371fa1b-871a-4fbc-a5df-c299d4464808 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:15.136217 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab79860-ba78-44a9-adfe-9b5d71867b07 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:15.136493 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab79860-ba78-44a9-adfe-9b5d71867b07 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:15.136765 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab79860-ba78-44a9-adfe-9b5d71867b07 None admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:15.137456 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab79860-ba78-44a9-adfe-9b5d71867b07 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:15.137764 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab79860-ba78-44a9-adfe-9b5d71867b07 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:15.139219 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab79860-ba78-44a9-adfe-9b5d71867b07 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:15.143865 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3371fa1b-871a-4fbc-a5df-c299d4464808 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:01:15.148529 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-3371fa1b-871a-4fbc-a5df-c299d4464808 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 25 03:01:15.148529 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:15.148529 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:15.148529 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:15.148529 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:15.148529 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:15.148529 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:15.148529 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:15.148529 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:15.148529 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:15.148529 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:15.148529 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:15.148529 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:01:15.148529 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:01:15.148529 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:01:15.148529 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:01:15.149573 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:01:15.149573 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:01:15.149573 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:01:15.149573 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:01:15.149573 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:15.149573 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:15.149573 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:15.149573 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:15.149573 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:15.149573 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:15.149573 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:15.149573 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:15.149573 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:15.149573 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:15.149573 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:15.149573 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:15.150454 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:15.150454 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:15.150454 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:15.150454 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:15.150454 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:01:15.150454 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:01:15.150454 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:01:15.150454 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:01:15.150454 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:01:15.150454 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:01:15.150454 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 25 03:01:15.150454 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:15.150454 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 332/663] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Aug 25 03:01:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:15.151174 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3371fa1b-871a-4fbc-a5df-c299d4464808 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:15.157941 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 332/664] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Aug 25 03:01:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:15.158825 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab79860-ba78-44a9-adfe-9b5d71867b07 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:15.249872 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3371fa1b-871a-4fbc-a5df-c299d4464808 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:15.254686 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab79860-ba78-44a9-adfe-9b5d71867b07 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:15.282939 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ff0929-b17a-4224-b387-1a03f156993f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:15.283537 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ff0929-b17a-4224-b387-1a03f156993f None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:15.283818 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ff0929-b17a-4224-b387-1a03f156993f None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:15.284100 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ff0929-b17a-4224-b387-1a03f156993f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:15.287264 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f2d8ca-5a54-453d-9129-7ec93337459b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:15.287858 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f2d8ca-5a54-453d-9129-7ec93337459b None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:15.288138 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f2d8ca-5a54-453d-9129-7ec93337459b None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:15.288411 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f2d8ca-5a54-453d-9129-7ec93337459b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:15.289126 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f2d8ca-5a54-453d-9129-7ec93337459b None admin] RBAC: Adding query filter params (role.id=a93ddb1fd6f6459fb46a17573b821422, scope.project.id=9fd4be43faec4e4191e0e4ec4a4588c3, user.id=0cd171bf4df94f8fab397e42835feeab) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:15.289431 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f2d8ca-5a54-453d-9129-7ec93337459b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:15.290871 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f2d8ca-5a54-453d-9129-7ec93337459b None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:15.296859 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 333/665] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Aug 25 03:01:15 2022] GET /identity/v3/role_assignments?user.id=0cd171bf4df94f8fab397e42835feeab&scope.project.id=9fd4be43faec4e4191e0e4ec4a4588c3&role.id=a93ddb1fd6f6459fb46a17573b821422 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:15.297627 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f2d8ca-5a54-453d-9129-7ec93337459b None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:15.324350 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:15.349563 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:15.361248 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ff0929-b17a-4224-b387-1a03f156993f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:15.362801 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ff0929-b17a-4224-b387-1a03f156993f None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:15.414125 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f2d8ca-5a54-453d-9129-7ec93337459b None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:15.460836 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a1fe35-f6c4-404d-8ca8-77ed9bc20ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:15.461759 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a1fe35-f6c4-404d-8ca8-77ed9bc20ee8 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:15.462212 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a1fe35-f6c4-404d-8ca8-77ed9bc20ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:15.462707 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a1fe35-f6c4-404d-8ca8-77ed9bc20ee8 None admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:15.463447 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a1fe35-f6c4-404d-8ca8-77ed9bc20ee8 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:15.463751 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a1fe35-f6c4-404d-8ca8-77ed9bc20ee8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:15.465189 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a1fe35-f6c4-404d-8ca8-77ed9bc20ee8 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:15.478000 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 333/666] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Aug 25 03:01:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 25 03:01:15.479116 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ff0929-b17a-4224-b387-1a03f156993f None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:15.486633 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 334/667] 10.222.0.58 () {56 vars in 1139 bytes} [Thu Aug 25 03:01:15 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:15.487568 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a1fe35-f6c4-404d-8ca8-77ed9bc20ee8 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:15.584502 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:15.587733 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:15.588272 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ff0929-b17a-4224-b387-1a03f156993f None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:15.612425 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a1fe35-f6c4-404d-8ca8-77ed9bc20ee8 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:15.625378 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937db97e-c6a1-4238-8222-1042bde6534f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:15.625994 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937db97e-c6a1-4238-8222-1042bde6534f None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:15.626274 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937db97e-c6a1-4238-8222-1042bde6534f None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:15.626576 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937db97e-c6a1-4238-8222-1042bde6534f None admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:15.627286 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937db97e-c6a1-4238-8222-1042bde6534f None admin] RBAC: Adding query filter params (name=service) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:15.627591 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937db97e-c6a1-4238-8222-1042bde6534f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:15.628980 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937db97e-c6a1-4238-8222-1042bde6534f None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:15.637749 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 334/668] 10.222.0.58 () {56 vars in 1103 bytes} [Thu Aug 25 03:01:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:15.638613 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937db97e-c6a1-4238-8222-1042bde6534f None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:15.660309 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474e023b-5f87-4d52-8eb8-3ce8c8ba64f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:15.661503 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474e023b-5f87-4d52-8eb8-3ce8c8ba64f3 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:15.661996 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474e023b-5f87-4d52-8eb8-3ce8c8ba64f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:15.662339 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474e023b-5f87-4d52-8eb8-3ce8c8ba64f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:15.663072 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474e023b-5f87-4d52-8eb8-3ce8c8ba64f3 None admin] RBAC: Adding query filter params (role.id=a93ddb1fd6f6459fb46a17573b821422, scope.project.id=9fd4be43faec4e4191e0e4ec4a4588c3, user.id=3f89089b390c4b6599e54000c0fa082a) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:15.663437 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474e023b-5f87-4d52-8eb8-3ce8c8ba64f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:15.664887 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474e023b-5f87-4d52-8eb8-3ce8c8ba64f3 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:15.672043 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 335/669] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Aug 25 03:01:15 2022] GET /identity/v3/role_assignments?user.id=3f89089b390c4b6599e54000c0fa082a&scope.project.id=9fd4be43faec4e4191e0e4ec4a4588c3&role.id=a93ddb1fd6f6459fb46a17573b821422 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:15.672847 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474e023b-5f87-4d52-8eb8-3ce8c8ba64f3 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:15.784287 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937db97e-c6a1-4238-8222-1042bde6534f None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:15.789197 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474e023b-5f87-4d52-8eb8-3ce8c8ba64f3 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:15.838989 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326dd4ab-1540-4423-9e53-a71e79c0ffbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:15.839612 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326dd4ab-1540-4423-9e53-a71e79c0ffbd None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:15.839895 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326dd4ab-1540-4423-9e53-a71e79c0ffbd None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:15.840783 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326dd4ab-1540-4423-9e53-a71e79c0ffbd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:15.841898 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76085db1-681c-4a78-8423-05a4b8f184a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:15.842635 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76085db1-681c-4a78-8423-05a4b8f184a8 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:15.842917 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76085db1-681c-4a78-8423-05a4b8f184a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:15.843188 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76085db1-681c-4a78-8423-05a4b8f184a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:15.857663 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76085db1-681c-4a78-8423-05a4b8f184a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:15.859616 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:15.860452 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76085db1-681c-4a78-8423-05a4b8f184a8 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:15.876836 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-76085db1-681c-4a78-8423-05a4b8f184a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:15.876836 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:15.876836 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:15.876836 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:15.876836 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:15.876836 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:15.876836 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:15.876836 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:15.876836 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:15.876836 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:15.876836 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:15.876836 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:15.876836 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 03:01:15.876836 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 03:01:15.876836 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 03:01:15.876836 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 03:01:15.877878 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:15.877878 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:15.877878 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:15.877878 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:15.877878 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:15.877878 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:15.877878 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:15.877878 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:15.877878 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:15.877878 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:15.877878 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:15.877878 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:15.877878 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:15.877878 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:15.877878 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:15.877878 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:15.877878 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 03:01:15.878833 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 03:01:15.878833 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:15.878833 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:15.880147 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 335/670] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:01:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:15.880329 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76085db1-681c-4a78-8423-05a4b8f184a8 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:15.891756 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:15.937673 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326dd4ab-1540-4423-9e53-a71e79c0ffbd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:15.939177 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326dd4ab-1540-4423-9e53-a71e79c0ffbd None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:16.002139 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76085db1-681c-4a78-8423-05a4b8f184a8 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:16.044461 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbda07c6-2829-47b0-b40a-9641576521a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:16.045620 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbda07c6-2829-47b0-b40a-9641576521a7 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:16.048135 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbda07c6-2829-47b0-b40a-9641576521a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:16.048484 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbda07c6-2829-47b0-b40a-9641576521a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:16.063059 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbda07c6-2829-47b0-b40a-9641576521a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:16.065157 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbda07c6-2829-47b0-b40a-9641576521a7 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:16.069934 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 336/671] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Aug 25 03:01:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 25 03:01:16.071108 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326dd4ab-1540-4423-9e53-a71e79c0ffbd None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:16.082262 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-dbda07c6-2829-47b0-b40a-9641576521a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:16.082262 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:16.082262 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:16.082262 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:16.082262 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:16.082262 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:16.082262 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:16.082262 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:16.082262 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:16.082262 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:16.082262 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:16.082262 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:16.082262 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 03:01:16.082262 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 03:01:16.082262 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 03:01:16.082262 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 03:01:16.083273 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:16.083273 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:16.083273 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:16.083273 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:16.083273 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:16.083273 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:16.083273 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:16.083273 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:16.083273 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:16.083273 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:16.083273 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:16.083273 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:16.083273 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:16.083273 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:16.083273 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:16.083273 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:16.083273 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 03:01:16.084244 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 03:01:16.084244 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:16.084244 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:16.086363 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 336/672] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:01:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:16.087429 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbda07c6-2829-47b0-b40a-9641576521a7 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:16.142440 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:16.153808 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:16.205246 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbda07c6-2829-47b0-b40a-9641576521a7 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:16.209902 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326dd4ab-1540-4423-9e53-a71e79c0ffbd None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:16.244347 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46721c3-e52f-414b-842d-1a954265ec1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:16.244968 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46721c3-e52f-414b-842d-1a954265ec1f None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:16.245270 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46721c3-e52f-414b-842d-1a954265ec1f None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:16.245530 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46721c3-e52f-414b-842d-1a954265ec1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:16.249515 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1123e17c-ad0e-4aec-a5ec-8d346937a099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:16.250519 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1123e17c-ad0e-4aec-a5ec-8d346937a099 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:16.251217 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1123e17c-ad0e-4aec-a5ec-8d346937a099 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:16.251917 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1123e17c-ad0e-4aec-a5ec-8d346937a099 None admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:16.253044 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1123e17c-ad0e-4aec-a5ec-8d346937a099 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:16.253558 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1123e17c-ad0e-4aec-a5ec-8d346937a099 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:16.255272 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1123e17c-ad0e-4aec-a5ec-8d346937a099 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:16.256603 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46721c3-e52f-414b-842d-1a954265ec1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:16.258043 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46721c3-e52f-414b-842d-1a954265ec1f None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:16.269283 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 337/673] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Aug 25 03:01:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:16.270644 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1123e17c-ad0e-4aec-a5ec-8d346937a099 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:16.273060 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-d46721c3-e52f-414b-842d-1a954265ec1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:16.273060 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:16.273060 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:16.273060 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:16.273060 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:16.273060 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:16.273060 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:16.273060 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:16.273060 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:16.273060 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:16.273060 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:16.273060 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:16.273060 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 03:01:16.273060 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 03:01:16.273060 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 03:01:16.273060 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 03:01:16.273998 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:16.273998 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:16.273998 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:16.273998 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:16.273998 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:16.273998 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:16.273998 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:16.273998 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:16.273998 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:16.273998 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:16.273998 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:16.273998 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:16.273998 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:16.273998 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:16.273998 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:16.273998 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:16.273998 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 03:01:16.274931 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 03:01:16.274931 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:16.274931 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:16.274931 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 337/674] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:01:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:16.275577 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46721c3-e52f-414b-842d-1a954265ec1f None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:16.384029 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1123e17c-ad0e-4aec-a5ec-8d346937a099 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:16.386776 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:16.412826 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46721c3-e52f-414b-842d-1a954265ec1f None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:16.412826 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:16.419607 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f08b50e-1910-403d-ac69-0c537999d799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:16.420491 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f08b50e-1910-403d-ac69-0c537999d799 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:16.421370 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f08b50e-1910-403d-ac69-0c537999d799 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:16.421719 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f08b50e-1910-403d-ac69-0c537999d799 None admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:16.422502 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f08b50e-1910-403d-ac69-0c537999d799 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:16.422967 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f08b50e-1910-403d-ac69-0c537999d799 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:16.424373 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f08b50e-1910-403d-ac69-0c537999d799 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:16.435859 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6773ecb-bf3d-4d0f-8f91-aeaa43e9b34f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:16.437287 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6773ecb-bf3d-4d0f-8f91-aeaa43e9b34f None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:16.437719 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6773ecb-bf3d-4d0f-8f91-aeaa43e9b34f None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:16.438148 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6773ecb-bf3d-4d0f-8f91-aeaa43e9b34f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:16.441708 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 338/675] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Aug 25 03:01:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:16.443117 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f08b50e-1910-403d-ac69-0c537999d799 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:16.455319 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6773ecb-bf3d-4d0f-8f91-aeaa43e9b34f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:16.457657 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6773ecb-bf3d-4d0f-8f91-aeaa43e9b34f None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:16.474472 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-f6773ecb-bf3d-4d0f-8f91-aeaa43e9b34f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 03:01:16.474472 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:16.474472 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:16.474472 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:16.474472 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:16.474472 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:16.474472 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:16.474472 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:16.474472 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:16.474472 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:16.474472 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:16.474472 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:16.474472 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 25 03:01:16.474472 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 25 03:01:16.474472 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 25 03:01:16.475395 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 25 03:01:16.475395 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:16.475395 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:16.475395 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:16.475395 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:16.475395 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:16.475395 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:16.475395 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:16.475395 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:16.475395 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:16.475395 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:16.475395 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:16.475395 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:16.475395 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:16.475395 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:16.475395 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:16.475395 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:16.476308 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 25 03:01:16.476308 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 25 03:01:16.476308 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 25 03:01:16.476308 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 25 03:01:16.476308 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 25 03:01:16.476308 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 25 03:01:16.476308 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 03:01:16.476308 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:16.477003 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 338/676] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Aug 25 03:01:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:16.479893 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6773ecb-bf3d-4d0f-8f91-aeaa43e9b34f None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:16.568824 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f08b50e-1910-403d-ac69-0c537999d799 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:16.586128 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6773ecb-bf3d-4d0f-8f91-aeaa43e9b34f None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:16.606542 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb30257-d8d4-4430-965a-eb650899cf41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:16.607592 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb30257-d8d4-4430-965a-eb650899cf41 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:16.608068 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb30257-d8d4-4430-965a-eb650899cf41 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:16.608567 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb30257-d8d4-4430-965a-eb650899cf41 None admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:16.609396 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb30257-d8d4-4430-965a-eb650899cf41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:16.609987 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb30257-d8d4-4430-965a-eb650899cf41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:16.611689 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb30257-d8d4-4430-965a-eb650899cf41 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:16.621425 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183cf978-f3a5-4a3d-9b23-cead241c0a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:16.622069 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183cf978-f3a5-4a3d-9b23-cead241c0a7a None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:16.622384 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183cf978-f3a5-4a3d-9b23-cead241c0a7a None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:16.622667 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183cf978-f3a5-4a3d-9b23-cead241c0a7a None admin] PATH_INFO: `/v3/users/swift` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:16.637113 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183cf978-f3a5-4a3d-9b23-cead241c0a7a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:16.638644 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183cf978-f3a5-4a3d-9b23-cead241c0a7a None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:16.651776 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-183cf978-f3a5-4a3d-9b23-cead241c0a7a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 25 03:01:16.651776 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:16.651776 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:16.651776 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:16.651776 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:16.651776 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:16.651776 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:16.651776 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:16.651776 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:16.651776 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:16.651776 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:16.651776 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:16.651776 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 25 03:01:16.651776 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 25 03:01:16.651776 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 25 03:01:16.651776 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 25 03:01:16.652792 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:16.652792 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:16.652792 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 25 03:01:16.652792 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 03:01:16.652792 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 25 03:01:16.652792 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 03:01:16.652792 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:16.652792 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:16.652792 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:16.652792 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:16.652792 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:16.652792 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:16.652792 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:16.652792 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:16.652792 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:16.652792 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:16.652792 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:16.652792 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:16.653722 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:16.653722 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:16.653722 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 25 03:01:16.653722 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 25 03:01:16.653722 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 25 03:01:16.653722 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 25 03:01:16.653722 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 25 03:01:16.653722 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 25 03:01:16.653722 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 25 03:01:16.653722 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:16.658392 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 339/677] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Aug 25 03:01:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:16.659571 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183cf978-f3a5-4a3d-9b23-cead241c0a7a None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:16.694068 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:16.749433 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 339/678] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Aug 25 03:01:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:16.750751 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb30257-d8d4-4430-965a-eb650899cf41 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:16.774489 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183cf978-f3a5-4a3d-9b23-cead241c0a7a None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:16.778966 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:16.812212 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9b84c9-4cdd-403b-859c-e4c974280811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:16.812833 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9b84c9-4cdd-403b-859c-e4c974280811 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:16.813118 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9b84c9-4cdd-403b-859c-e4c974280811 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:16.813388 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9b84c9-4cdd-403b-859c-e4c974280811 None admin] PATH_INFO: `/v3/projects` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:16.814084 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9b84c9-4cdd-403b-859c-e4c974280811 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:16.814413 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9b84c9-4cdd-403b-859c-e4c974280811 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:16.815853 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9b84c9-4cdd-403b-859c-e4c974280811 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:16.829904 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 340/679] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Aug 25 03:01:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:16.830563 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9b84c9-4cdd-403b-859c-e4c974280811 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:16.867197 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb30257-d8d4-4430-965a-eb650899cf41 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:16.905686 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b1f569-efcf-4526-bf4a-2d9bc8bc14e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:16.906708 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b1f569-efcf-4526-bf4a-2d9bc8bc14e8 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:16.907232 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b1f569-efcf-4526-bf4a-2d9bc8bc14e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:16.907696 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b1f569-efcf-4526-bf4a-2d9bc8bc14e8 None admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:16.908646 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b1f569-efcf-4526-bf4a-2d9bc8bc14e8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:16.909145 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b1f569-efcf-4526-bf4a-2d9bc8bc14e8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:16.910862 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b1f569-efcf-4526-bf4a-2d9bc8bc14e8 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:16.932507 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 340/680] 10.222.0.58 () {56 vars in 1135 bytes} [Thu Aug 25 03:01:16 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:16.933718 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b1f569-efcf-4526-bf4a-2d9bc8bc14e8 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:16.943768 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:16.957271 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9b84c9-4cdd-403b-859c-e4c974280811 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:17.005966 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96416a66-b583-4b1f-9119-64384c6431eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:17.006656 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96416a66-b583-4b1f-9119-64384c6431eb None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:17.006982 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96416a66-b583-4b1f-9119-64384c6431eb None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:17.007325 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96416a66-b583-4b1f-9119-64384c6431eb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:17.017365 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96416a66-b583-4b1f-9119-64384c6431eb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:17.019619 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96416a66-b583-4b1f-9119-64384c6431eb None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:17.033944 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-96416a66-b583-4b1f-9119-64384c6431eb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 03:01:17.033944 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:17.033944 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:17.033944 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:17.033944 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:17.033944 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:17.033944 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:17.033944 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:17.033944 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:17.033944 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:17.033944 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:17.033944 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:17.033944 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 25 03:01:17.033944 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 25 03:01:17.033944 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 25 03:01:17.035004 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 25 03:01:17.035004 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:17.035004 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:17.035004 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:17.035004 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:17.035004 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:17.035004 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:17.035004 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:17.035004 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:17.035004 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:17.035004 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:17.035004 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:17.035004 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:17.035004 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:17.035004 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:17.035004 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:17.035004 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:17.035971 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 25 03:01:17.035971 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 25 03:01:17.035971 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 25 03:01:17.035971 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 25 03:01:17.035971 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 25 03:01:17.035971 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 25 03:01:17.035971 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 03:01:17.035971 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:17.035971 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 341/681] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Aug 25 03:01:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:17.036461 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96416a66-b583-4b1f-9119-64384c6431eb None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:17.048420 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:17.074583 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b1f569-efcf-4526-bf4a-2d9bc8bc14e8 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:17.118685 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cc4767-9c11-4286-845e-9f3757c48b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:17.119652 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cc4767-9c11-4286-845e-9f3757c48b09 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:17.120136 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cc4767-9c11-4286-845e-9f3757c48b09 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:17.120625 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cc4767-9c11-4286-845e-9f3757c48b09 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:17.121542 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cc4767-9c11-4286-845e-9f3757c48b09 None admin] RBAC: Adding query filter params (role.id=a93ddb1fd6f6459fb46a17573b821422, scope.project.id=9fd4be43faec4e4191e0e4ec4a4588c3, user.id=60193ba9e1c14df3bc9eebcc6f28f2d7) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:17.122080 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cc4767-9c11-4286-845e-9f3757c48b09 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:17.123799 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cc4767-9c11-4286-845e-9f3757c48b09 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:17.130622 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 341/682] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Aug 25 03:01:16 2022] GET /identity/v3/role_assignments?user.id=60193ba9e1c14df3bc9eebcc6f28f2d7&scope.project.id=9fd4be43faec4e4191e0e4ec4a4588c3&role.id=a93ddb1fd6f6459fb46a17573b821422 => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:17.131750 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cc4767-9c11-4286-845e-9f3757c48b09 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:17.172740 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96416a66-b583-4b1f-9119-64384c6431eb None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:17.216165 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abe0e47-8208-4321-8705-cd73249441cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:17.217148 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abe0e47-8208-4321-8705-cd73249441cc None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:17.217674 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abe0e47-8208-4321-8705-cd73249441cc None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:17.218164 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abe0e47-8208-4321-8705-cd73249441cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:17.240578 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:17.260074 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cc4767-9c11-4286-845e-9f3757c48b09 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:17.301601 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abe0e47-8208-4321-8705-cd73249441cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:17.303539 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abe0e47-8208-4321-8705-cd73249441cc None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:17.314956 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36f6dc0-e9b1-4fef-9151-e614359cc6b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:17.315594 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36f6dc0-e9b1-4fef-9151-e614359cc6b0 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:17.315876 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36f6dc0-e9b1-4fef-9151-e614359cc6b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:17.316152 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36f6dc0-e9b1-4fef-9151-e614359cc6b0 None admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:17.316869 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36f6dc0-e9b1-4fef-9151-e614359cc6b0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:17.317180 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36f6dc0-e9b1-4fef-9151-e614359cc6b0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:17.319315 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36f6dc0-e9b1-4fef-9151-e614359cc6b0 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:17.335594 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 342/683] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Aug 25 03:01:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:17.337230 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5a8fe7-423d-484b-9c48-b9bc7d880775 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:17.337517 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5a8fe7-423d-484b-9c48-b9bc7d880775 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:17.338157 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5a8fe7-423d-484b-9c48-b9bc7d880775 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:17.342361 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 343/684] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:01:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:01:17.343519 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5a8fe7-423d-484b-9c48-b9bc7d880775 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:17.370644 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:17.435108 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 342/685] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Aug 25 03:01:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 25 03:01:17.437260 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4703c23-b369-419b-921f-2fb9a07e8ec7 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:17.437771 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4703c23-b369-419b-921f-2fb9a07e8ec7 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:17.438256 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4703c23-b369-419b-921f-2fb9a07e8ec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:17.462987 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-d4703c23-b369-419b-921f-2fb9a07e8ec7 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:01:17.464757 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-d4703c23-b369-419b-921f-2fb9a07e8ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:01:17.473476 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5a8fe7-423d-484b-9c48-b9bc7d880775 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:17.516910 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55504645-7147-4f32-b985-0e20e895b069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:17.517514 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55504645-7147-4f32-b985-0e20e895b069 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:17.517803 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55504645-7147-4f32-b985-0e20e895b069 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:17.518064 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55504645-7147-4f32-b985-0e20e895b069 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:17.518821 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55504645-7147-4f32-b985-0e20e895b069 None admin] RBAC: Adding query filter params (role.id=a93ddb1fd6f6459fb46a17573b821422, scope.project.id=9fd4be43faec4e4191e0e4ec4a4588c3, user.id=a0a1a7f091774842907fd08de15e8058) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:17.519125 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55504645-7147-4f32-b985-0e20e895b069 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:17.520690 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55504645-7147-4f32-b985-0e20e895b069 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:17.522855 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:17.529360 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 344/686] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Aug 25 03:01:17 2022] GET /identity/v3/role_assignments?user.id=a0a1a7f091774842907fd08de15e8058&scope.project.id=9fd4be43faec4e4191e0e4ec4a4588c3&role.id=a93ddb1fd6f6459fb46a17573b821422 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:17.536912 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55504645-7147-4f32-b985-0e20e895b069 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:17.548003 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 343/687] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:01:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 03:01:17.643083 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:17.675478 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55504645-7147-4f32-b985-0e20e895b069 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:17.747283 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cadb7f-6f8b-427d-9640-f61895da4fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:17.747940 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cadb7f-6f8b-427d-9640-f61895da4fd6 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:17.748221 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cadb7f-6f8b-427d-9640-f61895da4fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:17.748498 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cadb7f-6f8b-427d-9640-f61895da4fd6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:17.768050 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cadb7f-6f8b-427d-9640-f61895da4fd6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:17.769422 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cadb7f-6f8b-427d-9640-f61895da4fd6 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:17.788060 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-e6cadb7f-6f8b-427d-9640-f61895da4fd6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:17.788060 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:17.788060 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:17.788060 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:17.788060 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:17.788060 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:17.788060 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:17.788060 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:17.788060 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:17.788060 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:17.788060 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:17.788060 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:17.788060 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 03:01:17.788060 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 03:01:17.788060 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 03:01:17.788060 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 03:01:17.789034 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:17.789034 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:17.789034 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:17.789034 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:17.789034 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:17.789034 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:17.789034 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:17.789034 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:17.789034 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:17.789034 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:17.789034 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:17.789034 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:17.789034 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:17.789034 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:17.789034 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:17.789034 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:17.789034 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 03:01:17.789944 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 03:01:17.789944 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:17.789944 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:17.789944 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 345/688] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:01:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:17.795120 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4703c23-b369-419b-921f-2fb9a07e8ec7 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:17.886472 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cadb7f-6f8b-427d-9640-f61895da4fd6 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:17.919318 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4703c23-b369-419b-921f-2fb9a07e8ec7 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:17.971262 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eef0d6-f426-45f1-b938-02ab42c6100e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:17.971882 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eef0d6-f426-45f1-b938-02ab42c6100e None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:17.972163 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eef0d6-f426-45f1-b938-02ab42c6100e None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:17.972433 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eef0d6-f426-45f1-b938-02ab42c6100e None admin] PATH_INFO: `/v3/domains` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:17.973523 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16eef0d6-f426-45f1-b938-02ab42c6100e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:17.973833 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16eef0d6-f426-45f1-b938-02ab42c6100e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:17.975601 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16eef0d6-f426-45f1-b938-02ab42c6100e None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:17.991696 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 344/689] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Aug 25 03:01:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:17.993465 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba1573a-035d-4d75-8f9e-3cb6fb6449a1 None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:17.994915 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba1573a-035d-4d75-8f9e-3cb6fb6449a1 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:17.994915 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba1573a-035d-4d75-8f9e-3cb6fb6449a1 None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:18.000167 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 345/690] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:01:17 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:01:18.001500 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba1573a-035d-4d75-8f9e-3cb6fb6449a1 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:18.008531 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cadb7f-6f8b-427d-9640-f61895da4fd6 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:18.031426 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:18.047299 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777d95fa-1677-4ccd-93cc-c5ffff9fe237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:18.049137 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777d95fa-1677-4ccd-93cc-c5ffff9fe237 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:18.049628 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777d95fa-1677-4ccd-93cc-c5ffff9fe237 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:18.050080 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777d95fa-1677-4ccd-93cc-c5ffff9fe237 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:18.055300 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:18.066611 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777d95fa-1677-4ccd-93cc-c5ffff9fe237 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:01:18.074134 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-777d95fa-1677-4ccd-93cc-c5ffff9fe237 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 03:01:18.074134 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:18.074134 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:18.074134 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:18.074134 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:18.074134 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:18.074134 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:18.074134 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:18.074134 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:18.074134 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:18.074134 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:18.074134 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:18.074134 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:01:18.074134 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:01:18.074134 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:01:18.074134 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:01:18.076294 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:01:18.076294 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:01:18.076294 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:01:18.076294 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:01:18.076294 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:18.076294 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:18.076294 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:18.076294 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:18.076294 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:18.076294 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:18.076294 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:18.076294 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:18.076294 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:18.076294 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:18.076294 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:18.076294 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:18.077225 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:18.077225 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:18.077225 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:18.077225 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:18.077225 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:01:18.077225 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:01:18.077225 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:01:18.077225 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:01:18.077225 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:01:18.077225 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:01:18.077225 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 03:01:18.077225 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:18.077225 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 346/691] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Aug 25 03:01:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 03:01:18.078598 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda7b41a-73e4-4354-bd0c-91b07e513aa8 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:18.078865 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda7b41a-73e4-4354-bd0c-91b07e513aa8 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:18.080388 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda7b41a-73e4-4354-bd0c-91b07e513aa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:18.103398 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-bda7b41a-73e4-4354-bd0c-91b07e513aa8 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:01:18.104654 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-bda7b41a-73e4-4354-bd0c-91b07e513aa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:01:18.127229 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba1573a-035d-4d75-8f9e-3cb6fb6449a1 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:18.166862 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90103193-e2b0-4dce-88b3-6ae9e0f76d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:18.167917 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90103193-e2b0-4dce-88b3-6ae9e0f76d30 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:18.168445 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90103193-e2b0-4dce-88b3-6ae9e0f76d30 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:18.168944 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90103193-e2b0-4dce-88b3-6ae9e0f76d30 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:18.182955 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90103193-e2b0-4dce-88b3-6ae9e0f76d30 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:18.184869 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90103193-e2b0-4dce-88b3-6ae9e0f76d30 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:18.198305 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 347/692] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:01:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 03:01:18.198305 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda7b41a-73e4-4354-bd0c-91b07e513aa8 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:18.205627 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-90103193-e2b0-4dce-88b3-6ae9e0f76d30 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 03:01:18.205627 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:18.205627 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:18.205627 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:18.205627 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:18.205627 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:18.205627 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:18.205627 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:18.205627 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:18.205627 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:18.205627 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:18.205627 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:18.205627 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 25 03:01:18.205627 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 25 03:01:18.205627 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 25 03:01:18.206643 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 25 03:01:18.206643 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:18.206643 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:18.206643 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:18.206643 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:18.206643 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:18.206643 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:18.206643 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:18.206643 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:18.206643 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:18.206643 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:18.206643 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:18.206643 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:18.206643 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:18.206643 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:18.206643 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:18.206643 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:18.207535 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 25 03:01:18.207535 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 25 03:01:18.207535 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 25 03:01:18.207535 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 25 03:01:18.207535 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 25 03:01:18.207535 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 25 03:01:18.207535 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 03:01:18.207535 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:18.218833 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 346/693] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Aug 25 03:01:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:18.219882 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90103193-e2b0-4dce-88b3-6ae9e0f76d30 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:18.306247 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda7b41a-73e4-4354-bd0c-91b07e513aa8 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:18.343778 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764912d0-0eac-4536-a0ad-9bccf68f6c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:18.344657 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764912d0-0eac-4536-a0ad-9bccf68f6c25 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:18.345002 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764912d0-0eac-4536-a0ad-9bccf68f6c25 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:18.345283 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764912d0-0eac-4536-a0ad-9bccf68f6c25 None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:18.345956 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764912d0-0eac-4536-a0ad-9bccf68f6c25 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:18.346260 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764912d0-0eac-4536-a0ad-9bccf68f6c25 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:18.347770 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764912d0-0eac-4536-a0ad-9bccf68f6c25 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:18.356859 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 348/694] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Aug 25 03:01:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:18.357375 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:18.363088 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764912d0-0eac-4536-a0ad-9bccf68f6c25 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:18.397021 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:18.400121 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90103193-e2b0-4dce-88b3-6ae9e0f76d30 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:18.443555 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbcef8d-c254-4292-9c98-a00f26e2cf07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:18.444170 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbcef8d-c254-4292-9c98-a00f26e2cf07 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:18.444454 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbcef8d-c254-4292-9c98-a00f26e2cf07 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:18.444731 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbcef8d-c254-4292-9c98-a00f26e2cf07 None admin] PATH_INFO: `/v3/projects` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:18.445432 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbcef8d-c254-4292-9c98-a00f26e2cf07 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:18.445739 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbcef8d-c254-4292-9c98-a00f26e2cf07 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:18.447803 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbcef8d-c254-4292-9c98-a00f26e2cf07 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:18.488506 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 347/695] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Aug 25 03:01:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:18.494552 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbcef8d-c254-4292-9c98-a00f26e2cf07 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:18.498000 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764912d0-0eac-4536-a0ad-9bccf68f6c25 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:18.538417 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c7116d-8966-436e-b6b3-54690b2f0c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:18.539082 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c7116d-8966-436e-b6b3-54690b2f0c7f None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:18.539368 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c7116d-8966-436e-b6b3-54690b2f0c7f None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:18.539645 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c7116d-8966-436e-b6b3-54690b2f0c7f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:18.554882 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c7116d-8966-436e-b6b3-54690b2f0c7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:18.557044 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c7116d-8966-436e-b6b3-54690b2f0c7f None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:18.575612 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-a9c7116d-8966-436e-b6b3-54690b2f0c7f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:18.575612 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:18.575612 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:18.575612 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:18.575612 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:18.575612 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:18.575612 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:18.575612 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:18.575612 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:18.575612 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:18.575612 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:18.575612 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:18.575612 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 03:01:18.575612 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 03:01:18.575612 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 03:01:18.575612 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 03:01:18.576547 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:18.576547 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:18.576547 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:18.576547 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:18.576547 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:18.576547 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:18.576547 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:18.576547 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:18.576547 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:18.576547 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:18.576547 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:18.576547 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:18.576547 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:18.576547 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:18.576547 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:18.576547 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:18.576547 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 03:01:18.577507 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 03:01:18.577507 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:18.577507 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:18.577507 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 349/696] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:01:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 03:01:18.578460 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c7116d-8966-436e-b6b3-54690b2f0c7f None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:18.613376 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:18.643746 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbcef8d-c254-4292-9c98-a00f26e2cf07 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:18.688111 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbec298-9498-4588-8d03-57136022c277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:18.688725 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbec298-9498-4588-8d03-57136022c277 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:18.689007 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbec298-9498-4588-8d03-57136022c277 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:18.689286 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbec298-9498-4588-8d03-57136022c277 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:18.690025 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbec298-9498-4588-8d03-57136022c277 None admin] RBAC: Adding query filter params (role.id=a93ddb1fd6f6459fb46a17573b821422, scope.project.id=9fd4be43faec4e4191e0e4ec4a4588c3, user.id=5047e1a7cd2d416d90b2d8c5a6596522) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:18.690360 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbec298-9498-4588-8d03-57136022c277 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:18.692355 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbec298-9498-4588-8d03-57136022c277 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:18.706313 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 348/697] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Aug 25 03:01:18 2022] GET /identity/v3/role_assignments?user.id=5047e1a7cd2d416d90b2d8c5a6596522&scope.project.id=9fd4be43faec4e4191e0e4ec4a4588c3&role.id=a93ddb1fd6f6459fb46a17573b821422 => generated 608 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:18.706313 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbec298-9498-4588-8d03-57136022c277 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:18.713293 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c7116d-8966-436e-b6b3-54690b2f0c7f None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:18.734313 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:18.807705 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365efbd7-70e1-481a-88c6-be97d0888e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:18.808761 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365efbd7-70e1-481a-88c6-be97d0888e11 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:18.809264 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365efbd7-70e1-481a-88c6-be97d0888e11 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:18.809743 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365efbd7-70e1-481a-88c6-be97d0888e11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:18.826247 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbec298-9498-4588-8d03-57136022c277 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:18.836581 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365efbd7-70e1-481a-88c6-be97d0888e11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:18.838447 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365efbd7-70e1-481a-88c6-be97d0888e11 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:18.868051 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ce87b4-7ee3-4047-981d-918111e824f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:18.868907 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce87b4-7ee3-4047-981d-918111e824f9 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:18.869194 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce87b4-7ee3-4047-981d-918111e824f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:18.869472 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce87b4-7ee3-4047-981d-918111e824f9 None admin] PATH_INFO: `/v3/domains` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:18.870130 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ce87b4-7ee3-4047-981d-918111e824f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:18.870537 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ce87b4-7ee3-4047-981d-918111e824f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:18.871935 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ce87b4-7ee3-4047-981d-918111e824f9 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:18.874531 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-365efbd7-70e1-481a-88c6-be97d0888e11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:18.874531 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:18.874531 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:18.874531 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:18.874531 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:18.874531 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:18.874531 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:18.874531 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:18.874531 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:18.874531 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:18.874531 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:18.874531 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:18.874531 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 03:01:18.874531 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 03:01:18.874531 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 03:01:18.874531 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 03:01:18.875443 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:18.875443 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:18.875443 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:18.875443 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:18.875443 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:18.875443 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:18.875443 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:18.875443 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:18.875443 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:18.875443 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:18.875443 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:18.875443 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:18.875443 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:18.875443 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:18.875443 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:18.875443 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:18.875443 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 03:01:18.876317 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 03:01:18.876317 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:18.876317 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:18.878414 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 350/698] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:01:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:18.884378 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365efbd7-70e1-481a-88c6-be97d0888e11 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:18.884773 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 349/699] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Aug 25 03:01:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:18.892124 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ce87b4-7ee3-4047-981d-918111e824f9 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:18.991671 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:19.005955 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:19.019427 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365efbd7-70e1-481a-88c6-be97d0888e11 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:19.032922 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ce87b4-7ee3-4047-981d-918111e824f9 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:19.061118 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f71eef3-aff7-47f5-a122-04e3b1a3a3cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:19.062850 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f71eef3-aff7-47f5-a122-04e3b1a3a3cc None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:19.063723 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f71eef3-aff7-47f5-a122-04e3b1a3a3cc None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:19.064462 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f71eef3-aff7-47f5-a122-04e3b1a3a3cc None admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:19.065749 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f71eef3-aff7-47f5-a122-04e3b1a3a3cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:19.066257 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f71eef3-aff7-47f5-a122-04e3b1a3a3cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:19.068644 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f71eef3-aff7-47f5-a122-04e3b1a3a3cc None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:19.075867 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f559f404-aa7c-46fa-ac95-6e45f3668f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:19.076566 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f559f404-aa7c-46fa-ac95-6e45f3668f15 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:19.076849 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f559f404-aa7c-46fa-ac95-6e45f3668f15 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:19.077974 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f559f404-aa7c-46fa-ac95-6e45f3668f15 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:19.086874 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 351/700] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Aug 25 03:01:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:19.092603 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f559f404-aa7c-46fa-ac95-6e45f3668f15 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:19.092603 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f559f404-aa7c-46fa-ac95-6e45f3668f15 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:19.093928 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f71eef3-aff7-47f5-a122-04e3b1a3a3cc None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:19.102654 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-f559f404-aa7c-46fa-ac95-6e45f3668f15 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 25 03:01:19.102654 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:19.102654 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:19.102654 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:19.102654 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:19.102654 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:19.102654 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:19.102654 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:19.102654 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:19.102654 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:19.102654 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:19.102654 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:19.102654 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 25 03:01:19.102654 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 25 03:01:19.102654 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 25 03:01:19.102654 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 25 03:01:19.103602 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:19.103602 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:19.103602 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 25 03:01:19.103602 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 03:01:19.103602 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 25 03:01:19.103602 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 03:01:19.103602 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:19.103602 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:19.103602 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:19.103602 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:19.103602 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:19.103602 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:19.103602 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:19.103602 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:19.103602 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:19.103602 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:19.103602 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:19.103602 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:19.104540 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:19.104540 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:19.104540 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 25 03:01:19.104540 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 25 03:01:19.104540 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 25 03:01:19.104540 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 25 03:01:19.104540 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 25 03:01:19.104540 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 25 03:01:19.104540 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 25 03:01:19.104540 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:19.107123 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 350/701] 10.222.0.58 () {56 vars in 1093 bytes} [Thu Aug 25 03:01:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:19.113874 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f559f404-aa7c-46fa-ac95-6e45f3668f15 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:19.217543 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f71eef3-aff7-47f5-a122-04e3b1a3a3cc None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:19.236497 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f559f404-aa7c-46fa-ac95-6e45f3668f15 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:19.256855 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505830e4-6b3e-43c9-b5f8-37ded3cf11a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:19.257800 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505830e4-6b3e-43c9-b5f8-37ded3cf11a3 None admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:19.258312 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505830e4-6b3e-43c9-b5f8-37ded3cf11a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:19.258790 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505830e4-6b3e-43c9-b5f8-37ded3cf11a3 None admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:19.259625 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505830e4-6b3e-43c9-b5f8-37ded3cf11a3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:19.261244 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505830e4-6b3e-43c9-b5f8-37ded3cf11a3 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:19.272421 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:19.278954 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36d04c2-53e1-4d51-bc75-0c2a49b414ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:19.279296 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36d04c2-53e1-4d51-bc75-0c2a49b414ac None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:19.279510 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36d04c2-53e1-4d51-bc75-0c2a49b414ac None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:19.279782 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36d04c2-53e1-4d51-bc75-0c2a49b414ac None admin] PATH_INFO: `/v3/users` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:19.280571 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36d04c2-53e1-4d51-bc75-0c2a49b414ac None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:19.280927 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36d04c2-53e1-4d51-bc75-0c2a49b414ac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:19.282336 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36d04c2-53e1-4d51-bc75-0c2a49b414ac None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:19.304315 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 351/702] 10.222.0.58 () {56 vars in 1133 bytes} [Thu Aug 25 03:01:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:19.309823 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36d04c2-53e1-4d51-bc75-0c2a49b414ac None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:19.314339 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:19.365278 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 352/703] 10.222.0.58 () {60 vars in 1132 bytes} [Thu Aug 25 03:01:19 2022] POST /identity/v3/users => generated 254 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:01:19.443800 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e20aa6a-ea5f-4475-aba5-f9b6db217422 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:19.444324 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e20aa6a-ea5f-4475-aba5-f9b6db217422 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:19.444746 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e20aa6a-ea5f-4475-aba5-f9b6db217422 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:19.446443 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 353/704] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:01:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 25 03:01:19.451543 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36d04c2-53e1-4d51-bc75-0c2a49b414ac None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:19.452824 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df1ce11-c628-4885-aada-135a14d382d6 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:19.453113 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df1ce11-c628-4885-aada-135a14d382d6 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:19.453391 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df1ce11-c628-4885-aada-135a14d382d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:19.488213 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-1df1ce11-c628-4885-aada-135a14d382d6 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:01:19.489442 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-1df1ce11-c628-4885-aada-135a14d382d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:01:19.514418 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c715da-86db-40e4-b386-6ec518c81b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:19.515102 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c715da-86db-40e4-b386-6ec518c81b2b None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:19.515382 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c715da-86db-40e4-b386-6ec518c81b2b None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:19.515658 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c715da-86db-40e4-b386-6ec518c81b2b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:19.618853 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 354/705] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:01:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 25 03:01:19.629777 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c715da-86db-40e4-b386-6ec518c81b2b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:19.630083 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c715da-86db-40e4-b386-6ec518c81b2b None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:19.633986 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:19.735672 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 352/706] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Aug 25 03:01:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 25 03:01:19.741024 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df1ce11-c628-4885-aada-135a14d382d6 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:19.843571 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df1ce11-c628-4885-aada-135a14d382d6 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:19.867763 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:19.879193 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aceeecb-1af3-4570-a604-b304f8cc6c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:19.879744 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aceeecb-1af3-4570-a604-b304f8cc6c0f None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:19.880024 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aceeecb-1af3-4570-a604-b304f8cc6c0f None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:19.880300 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aceeecb-1af3-4570-a604-b304f8cc6c0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:19.891871 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aceeecb-1af3-4570-a604-b304f8cc6c0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:19.895613 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aceeecb-1af3-4570-a604-b304f8cc6c0f None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:19.908793 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-2aceeecb-1af3-4570-a604-b304f8cc6c0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:19.908793 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:19.908793 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:19.908793 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:19.908793 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:19.908793 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:19.908793 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:19.908793 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:19.908793 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:19.908793 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:19.908793 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:19.908793 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:19.908793 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 03:01:19.908793 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 03:01:19.908793 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 03:01:19.908793 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 03:01:19.909862 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:19.909862 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:19.909862 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:19.909862 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:19.909862 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:19.909862 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:19.909862 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:19.909862 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:19.909862 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:19.909862 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:19.909862 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:19.909862 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:19.909862 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:19.909862 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:19.909862 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:19.909862 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:19.909862 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 03:01:19.910786 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 03:01:19.910786 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:19.910786 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:19.910786 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 355/707] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:01:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 03:01:19.919447 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c715da-86db-40e4-b386-6ec518c81b2b None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:19.945137 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aceeecb-1af3-4570-a604-b304f8cc6c0f None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:20.042260 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c715da-86db-40e4-b386-6ec518c81b2b None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:20.053303 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:20.082255 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79f3f65-401e-49cd-8854-bb493466e1bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:20.082901 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79f3f65-401e-49cd-8854-bb493466e1bb None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:20.083178 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79f3f65-401e-49cd-8854-bb493466e1bb None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:20.083400 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79f3f65-401e-49cd-8854-bb493466e1bb None admin] PATH_INFO: `/v3/domains` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:20.084230 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79f3f65-401e-49cd-8854-bb493466e1bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:20.085362 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79f3f65-401e-49cd-8854-bb493466e1bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:20.086788 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79f3f65-401e-49cd-8854-bb493466e1bb None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:20.099674 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 353/708] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Aug 25 03:01:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:20.105697 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79f3f65-401e-49cd-8854-bb493466e1bb None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:20.140255 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aceeecb-1af3-4570-a604-b304f8cc6c0f None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:20.184844 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c30786-1996-4581-94ed-8f444fcbeebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:20.185497 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c30786-1996-4581-94ed-8f444fcbeebf None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:20.185781 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c30786-1996-4581-94ed-8f444fcbeebf None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:20.186056 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c30786-1996-4581-94ed-8f444fcbeebf None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:20.187268 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c30786-1996-4581-94ed-8f444fcbeebf None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:20.188639 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c30786-1996-4581-94ed-8f444fcbeebf None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:20.202798 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-51c30786-1996-4581-94ed-8f444fcbeebf None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 25 03:01:20.202798 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:20.202798 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 25 03:01:20.202798 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 25 03:01:20.202798 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 25 03:01:20.202798 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 25 03:01:20.202798 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 25 03:01:20.202798 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 03:01:20.202798 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 25 03:01:20.202798 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:20.202798 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 25 03:01:20.202798 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:20.202798 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:20.202798 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:20.202798 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:20.202798 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:20.202798 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:20.204325 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:20.204325 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:20.204325 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:20.204325 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:20.204325 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:20.204325 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:20.204325 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 25 03:01:20.204325 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 25 03:01:20.204325 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 25 03:01:20.204325 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 25 03:01:20.204325 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:20.204325 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:20.204325 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:20.204325 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:20.204325 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:20.204325 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:20.205179 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:20.205179 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:20.205179 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:20.205179 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:20.205179 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:20.205179 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:20.205179 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:20.205179 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:20.205179 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:20.205179 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:20.205179 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 25 03:01:20.205179 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 03:01:20.205179 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 25 03:01:20.205179 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:20.205179 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 356/709] 10.222.0.58 () {56 vars in 1129 bytes} [Thu Aug 25 03:01:19 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 03:01:20.206554 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9829a7-6c83-4441-abbb-96dccd624e29 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:20.206554 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9829a7-6c83-4441-abbb-96dccd624e29 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:20.206849 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9829a7-6c83-4441-abbb-96dccd624e29 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:20.210343 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 357/710] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:01:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:01:20.212496 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9829a7-6c83-4441-abbb-96dccd624e29 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:20.230258 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79f3f65-401e-49cd-8854-bb493466e1bb None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:20.268043 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3501f436-2c86-4598-876c-05e4ea6115d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:20.268709 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3501f436-2c86-4598-876c-05e4ea6115d5 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:20.268996 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3501f436-2c86-4598-876c-05e4ea6115d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:20.269272 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3501f436-2c86-4598-876c-05e4ea6115d5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:20.271951 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:20.276538 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3501f436-2c86-4598-876c-05e4ea6115d5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:20.278062 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3501f436-2c86-4598-876c-05e4ea6115d5 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:20.290161 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-3501f436-2c86-4598-876c-05e4ea6115d5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 03:01:20.290161 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:20.290161 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:20.290161 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:20.290161 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:20.290161 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:20.290161 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:20.290161 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:20.290161 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:20.290161 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:20.290161 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:20.290161 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:20.290161 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 25 03:01:20.290161 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 25 03:01:20.290161 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 25 03:01:20.291333 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 25 03:01:20.291333 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:20.291333 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:20.291333 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:20.291333 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:20.291333 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:20.291333 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:20.291333 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:20.291333 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:20.291333 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:20.291333 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:20.291333 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:20.291333 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:20.291333 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:20.291333 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:20.291333 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:20.291333 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:20.292447 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 25 03:01:20.292447 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 25 03:01:20.292447 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 25 03:01:20.292447 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 25 03:01:20.292447 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 25 03:01:20.292447 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 25 03:01:20.292447 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 03:01:20.292447 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:20.296559 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 354/711] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Aug 25 03:01:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:20.297887 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c96873-9fb4-48e8-8311-c456a6ced105 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:20.298177 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c96873-9fb4-48e8-8311-c456a6ced105 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:20.298479 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c96873-9fb4-48e8-8311-c456a6ced105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:20.318733 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-80c96873-9fb4-48e8-8311-c456a6ced105 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:01:20.320034 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-80c96873-9fb4-48e8-8311-c456a6ced105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:01:20.338570 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:20.348118 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9829a7-6c83-4441-abbb-96dccd624e29 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:20.395899 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 355/712] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:01:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 03:01:20.396279 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960b32e9-32cc-4902-8616-38691d3c9856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:20.396965 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c96873-9fb4-48e8-8311-c456a6ced105 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:20.397287 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960b32e9-32cc-4902-8616-38691d3c9856 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:20.397801 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960b32e9-32cc-4902-8616-38691d3c9856 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:20.398218 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960b32e9-32cc-4902-8616-38691d3c9856 None admin] PATH_INFO: `/v3/services` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:20.399024 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960b32e9-32cc-4902-8616-38691d3c9856 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:20.399754 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960b32e9-32cc-4902-8616-38691d3c9856 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:20.402216 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960b32e9-32cc-4902-8616-38691d3c9856 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:20.412936 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 358/713] 10.222.0.58 () {56 vars in 1124 bytes} [Thu Aug 25 03:01:20 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:01:20.414482 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960b32e9-32cc-4902-8616-38691d3c9856 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:20.521737 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960b32e9-32cc-4902-8616-38691d3c9856 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:20.524850 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:20.532690 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c96873-9fb4-48e8-8311-c456a6ced105 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:20.557835 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fb497e-3a1a-4b23-96e2-0453876e1a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:20.558525 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fb497e-3a1a-4b23-96e2-0453876e1a36 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:20.558808 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fb497e-3a1a-4b23-96e2-0453876e1a36 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:20.559084 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fb497e-3a1a-4b23-96e2-0453876e1a36 None admin] PATH_INFO: `/v3/services` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:20.559867 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fb497e-3a1a-4b23-96e2-0453876e1a36 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:20.559995 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fb497e-3a1a-4b23-96e2-0453876e1a36 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:20.561481 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fb497e-3a1a-4b23-96e2-0453876e1a36 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:20.570320 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 359/714] 10.222.0.58 () {56 vars in 1124 bytes} [Thu Aug 25 03:01:20 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:01:20.596041 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6d6314-ef1a-443c-ae51-438b159877e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:20.596823 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6d6314-ef1a-443c-ae51-438b159877e5 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:20.597204 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6d6314-ef1a-443c-ae51-438b159877e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:20.597593 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6d6314-ef1a-443c-ae51-438b159877e5 None admin] PATH_INFO: `/v3/projects` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:20.598435 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6d6314-ef1a-443c-ae51-438b159877e5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:20.598839 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6d6314-ef1a-443c-ae51-438b159877e5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:20.600370 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6d6314-ef1a-443c-ae51-438b159877e5 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:20.624821 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 356/715] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Aug 25 03:01:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:20.630494 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fb497e-3a1a-4b23-96e2-0453876e1a36 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:20.787218 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fb497e-3a1a-4b23-96e2-0453876e1a36 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:20.824821 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6ea38a-0287-448d-a2f6-9a4ddf60abe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:20.825444 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6ea38a-0287-448d-a2f6-9a4ddf60abe9 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:20.825724 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6ea38a-0287-448d-a2f6-9a4ddf60abe9 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:20.825996 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6ea38a-0287-448d-a2f6-9a4ddf60abe9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:20.826765 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6ea38a-0287-448d-a2f6-9a4ddf60abe9 None admin] RBAC: Adding query filter params (role.id=62f19ea272b447ccaab1576cd9bf4fd9, scope.project.id=9fd4be43faec4e4191e0e4ec4a4588c3, user.id=0cd171bf4df94f8fab397e42835feeab) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:20.827876 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6ea38a-0287-448d-a2f6-9a4ddf60abe9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:20.829306 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6ea38a-0287-448d-a2f6-9a4ddf60abe9 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:20.835397 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 360/716] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Aug 25 03:01:20 2022] GET /identity/v3/role_assignments?user.id=0cd171bf4df94f8fab397e42835feeab&scope.project.id=9fd4be43faec4e4191e0e4ec4a4588c3&role.id=62f19ea272b447ccaab1576cd9bf4fd9 => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:20.838878 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6d6314-ef1a-443c-ae51-438b159877e5 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:20.857870 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:21.088200 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6d6314-ef1a-443c-ae51-438b159877e5 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:21.123952 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94426fb-49be-43f0-9208-3242f90d4051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:21.125115 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94426fb-49be-43f0-9208-3242f90d4051 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:21.125904 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94426fb-49be-43f0-9208-3242f90d4051 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:21.126193 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94426fb-49be-43f0-9208-3242f90d4051 None admin] PATH_INFO: `/v3/services/network` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:21.126827 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94426fb-49be-43f0-9208-3242f90d4051 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:21.128282 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94426fb-49be-43f0-9208-3242f90d4051 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:21.139689 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-c94426fb-49be-43f0-9208-3242f90d4051 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 25 03:01:21.139689 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:21.139689 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 25 03:01:21.139689 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 25 03:01:21.139689 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 25 03:01:21.139689 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 25 03:01:21.139689 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 25 03:01:21.139689 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 03:01:21.139689 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 25 03:01:21.139689 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:21.139689 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 25 03:01:21.139689 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:21.139689 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:21.139689 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:21.139689 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:21.139689 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:21.139689 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:21.140715 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:21.140715 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:21.140715 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:21.140715 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:21.140715 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:21.140715 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:21.140715 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 25 03:01:21.140715 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 25 03:01:21.140715 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 25 03:01:21.140715 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 25 03:01:21.140715 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:21.140715 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:21.140715 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:21.140715 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:21.140715 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:21.140715 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:21.141540 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:21.141540 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:21.141540 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:21.141540 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:21.141540 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:21.141540 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:21.141540 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:21.141540 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:21.141540 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:21.141540 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:21.141540 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 25 03:01:21.141540 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 03:01:21.141540 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 25 03:01:21.141540 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:21.141540 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 357/717] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Aug 25 03:01:20 2022] GET /identity/v3/services/network => generated 88 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:21.147519 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6ea38a-0287-448d-a2f6-9a4ddf60abe9 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:21.179667 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:21.267559 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6ea38a-0287-448d-a2f6-9a4ddf60abe9 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:21.313460 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd512a91-be33-4007-b3db-543a82ec9db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:21.314082 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd512a91-be33-4007-b3db-543a82ec9db2 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:21.314389 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd512a91-be33-4007-b3db-543a82ec9db2 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:21.314663 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd512a91-be33-4007-b3db-543a82ec9db2 None admin] PATH_INFO: `/v3/services` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:21.315305 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd512a91-be33-4007-b3db-543a82ec9db2 None admin] RBAC: Adding query filter params (name=network) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:21.315607 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd512a91-be33-4007-b3db-543a82ec9db2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:21.316972 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd512a91-be33-4007-b3db-543a82ec9db2 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:21.325777 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 361/718] 10.222.0.58 () {56 vars in 1112 bytes} [Thu Aug 25 03:01:21 2022] GET /identity/v3/services?name=network => generated 124 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:21.329751 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94426fb-49be-43f0-9208-3242f90d4051 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:21.352768 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbe1c59-4b68-4b63-ab5c-4a607b9da912 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:21.352768 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbe1c59-4b68-4b63-ab5c-4a607b9da912 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:21.352768 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbe1c59-4b68-4b63-ab5c-4a607b9da912 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:21.354034 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 362/719] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:01:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:01:21.365079 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac504b7-b286-4962-9103-7a9f96f51503 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:21.365355 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac504b7-b286-4962-9103-7a9f96f51503 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:21.365625 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac504b7-b286-4962-9103-7a9f96f51503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:21.399635 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-3ac504b7-b286-4962-9103-7a9f96f51503 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:01:21.400842 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-3ac504b7-b286-4962-9103-7a9f96f51503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:01:21.434638 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:21.479469 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94426fb-49be-43f0-9208-3242f90d4051 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:21.535680 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 363/720] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:01:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 25 03:01:21.549924 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6303ec67-f8d0-4f64-8468-00a09ac7ebe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:21.550576 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6303ec67-f8d0-4f64-8468-00a09ac7ebe3 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:21.550860 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6303ec67-f8d0-4f64-8468-00a09ac7ebe3 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:21.551135 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6303ec67-f8d0-4f64-8468-00a09ac7ebe3 None admin] PATH_INFO: `/v3/services` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:21.551778 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6303ec67-f8d0-4f64-8468-00a09ac7ebe3 None admin] RBAC: Adding query filter params (type=network) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:21.552080 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6303ec67-f8d0-4f64-8468-00a09ac7ebe3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:21.553442 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6303ec67-f8d0-4f64-8468-00a09ac7ebe3 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:21.564184 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 358/721] 10.222.0.58 () {56 vars in 1112 bytes} [Thu Aug 25 03:01:21 2022] GET /identity/v3/services?type=network => generated 124 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:21.838417 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac504b7-b286-4962-9103-7a9f96f51503 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:21.852783 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abad1764-d823-48e8-8ef2-be44861a3bb0 None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:21.853093 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abad1764-d823-48e8-8ef2-be44861a3bb0 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:21.853351 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abad1764-d823-48e8-8ef2-be44861a3bb0 None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:21.854931 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 359/722] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:01:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:01:21.860177 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b9dc46-7508-48c5-94e0-d6ac5baac212 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:21.860416 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b9dc46-7508-48c5-94e0-d6ac5baac212 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:21.860692 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b9dc46-7508-48c5-94e0-d6ac5baac212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:21.866885 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:21.890753 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-a7b9dc46-7508-48c5-94e0-d6ac5baac212 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:01:21.892058 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-a7b9dc46-7508-48c5-94e0-d6ac5baac212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:01:21.967594 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 360/723] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:01:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 25 03:01:21.988695 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac504b7-b286-4962-9103-7a9f96f51503 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:22.025663 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0559d83-dee1-429a-b56c-001443dc7573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:22.026334 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0559d83-dee1-429a-b56c-001443dc7573 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:22.026621 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0559d83-dee1-429a-b56c-001443dc7573 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:22.026945 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0559d83-dee1-429a-b56c-001443dc7573 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:22.037982 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0559d83-dee1-429a-b56c-001443dc7573 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:01:22.042821 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-f0559d83-dee1-429a-b56c-001443dc7573 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 03:01:22.042821 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:22.042821 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:22.042821 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:22.042821 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:22.042821 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:22.042821 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:22.042821 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:22.042821 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:22.042821 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:22.042821 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:22.042821 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:22.042821 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:01:22.042821 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:01:22.042821 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:01:22.042821 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:01:22.043739 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:01:22.043739 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:01:22.043739 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:01:22.043739 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:01:22.043739 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:22.043739 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:22.043739 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:22.043739 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:22.043739 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:22.043739 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:22.043739 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:22.043739 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:22.043739 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:22.043739 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:22.043739 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:22.043739 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:22.044727 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:22.044727 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:22.044727 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:22.044727 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:22.044727 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:01:22.044727 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:01:22.044727 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:01:22.044727 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:01:22.044727 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:01:22.044727 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:01:22.044727 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 03:01:22.044727 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:22.044727 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 364/724] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Aug 25 03:01:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:22.050050 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b9dc46-7508-48c5-94e0-d6ac5baac212 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:22.158373 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:22.200255 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b9dc46-7508-48c5-94e0-d6ac5baac212 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:22.252686 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195a8110-6a3c-4761-852d-cdadce4391cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:22.253307 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195a8110-6a3c-4761-852d-cdadce4391cc None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:22.253587 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195a8110-6a3c-4761-852d-cdadce4391cc None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:22.253858 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195a8110-6a3c-4761-852d-cdadce4391cc None admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:22.254586 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195a8110-6a3c-4761-852d-cdadce4391cc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:22.254860 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195a8110-6a3c-4761-852d-cdadce4391cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:22.256239 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195a8110-6a3c-4761-852d-cdadce4391cc None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:22.270506 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 361/725] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Aug 25 03:01:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:01:22.276010 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0559d83-dee1-429a-b56c-001443dc7573 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:22.277546 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195a8110-6a3c-4761-852d-cdadce4391cc None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:22.402029 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:22.419096 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0559d83-dee1-429a-b56c-001443dc7573 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:22.458039 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a22124-78ea-4969-bb94-a0b3fba00b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:22.458723 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a22124-78ea-4969-bb94-a0b3fba00b22 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:22.459014 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a22124-78ea-4969-bb94-a0b3fba00b22 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:22.459280 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a22124-78ea-4969-bb94-a0b3fba00b22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:22.470687 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a22124-78ea-4969-bb94-a0b3fba00b22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:22.472153 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a22124-78ea-4969-bb94-a0b3fba00b22 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:22.488468 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-49a22124-78ea-4969-bb94-a0b3fba00b22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:22.488468 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:22.488468 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:22.488468 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:22.488468 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:22.488468 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:22.488468 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:22.488468 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:22.488468 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:22.488468 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:22.488468 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:22.488468 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:22.488468 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 03:01:22.488468 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 03:01:22.488468 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 03:01:22.488468 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 03:01:22.489450 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:22.489450 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:22.489450 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:22.489450 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:22.489450 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:22.489450 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:22.489450 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:22.489450 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:22.489450 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:22.489450 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:22.489450 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:22.489450 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:22.489450 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:22.489450 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:22.489450 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:22.489450 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:22.489450 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 03:01:22.490940 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 03:01:22.490940 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:22.490940 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:22.490940 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 365/726] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:01:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:22.495115 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a22124-78ea-4969-bb94-a0b3fba00b22 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:22.560230 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195a8110-6a3c-4761-852d-cdadce4391cc None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:22.603863 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a22124-78ea-4969-bb94-a0b3fba00b22 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:22.647053 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:22.651726 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7908f05a-a9c5-4ba2-96ad-412fe0feac07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:22.652352 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7908f05a-a9c5-4ba2-96ad-412fe0feac07 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:22.659437 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7908f05a-a9c5-4ba2-96ad-412fe0feac07 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:22.659437 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7908f05a-a9c5-4ba2-96ad-412fe0feac07 None admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:22.659437 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7908f05a-a9c5-4ba2-96ad-412fe0feac07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:22.659437 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7908f05a-a9c5-4ba2-96ad-412fe0feac07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:22.659437 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7908f05a-a9c5-4ba2-96ad-412fe0feac07 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:22.659792 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07955e2-5886-4bf8-a268-dde11bdf2c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:22.659792 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07955e2-5886-4bf8-a268-dde11bdf2c85 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:22.659792 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07955e2-5886-4bf8-a268-dde11bdf2c85 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:22.659792 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07955e2-5886-4bf8-a268-dde11bdf2c85 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:22.671675 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:22.680875 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 366/727] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Aug 25 03:01:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:01:22.681242 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07955e2-5886-4bf8-a268-dde11bdf2c85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:01:22.687147 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7908f05a-a9c5-4ba2-96ad-412fe0feac07 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:22.687859 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-a07955e2-5886-4bf8-a268-dde11bdf2c85 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 25 03:01:22.687859 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:22.687859 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:22.687859 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:22.687859 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:22.687859 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:22.687859 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:22.687859 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:22.687859 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:22.687859 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:22.687859 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:22.687859 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:22.687859 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:01:22.687859 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:01:22.687859 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:01:22.687859 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:01:22.688767 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:01:22.688767 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:01:22.688767 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:01:22.688767 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:01:22.688767 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:22.688767 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:22.688767 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:22.688767 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:22.688767 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:22.688767 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:22.688767 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:22.688767 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:22.688767 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:22.688767 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:22.688767 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:22.688767 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:22.689619 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:22.689619 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:22.689619 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:22.689619 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:22.689619 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:01:22.689619 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:01:22.689619 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:01:22.689619 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:01:22.689619 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:01:22.689619 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:01:22.689619 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 25 03:01:22.689619 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:22.689619 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 362/728] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Aug 25 03:01:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 412 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 03:01:22.697987 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07955e2-5886-4bf8-a268-dde11bdf2c85 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:22.828628 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07955e2-5886-4bf8-a268-dde11bdf2c85 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:22.832328 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7908f05a-a9c5-4ba2-96ad-412fe0feac07 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:22.864080 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed951b2-48e5-4c21-844f-60b9c4d537a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:22.864692 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed951b2-48e5-4c21-844f-60b9c4d537a0 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:22.864972 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed951b2-48e5-4c21-844f-60b9c4d537a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:22.865249 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed951b2-48e5-4c21-844f-60b9c4d537a0 None admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:22.865991 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed951b2-48e5-4c21-844f-60b9c4d537a0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:22.866212 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed951b2-48e5-4c21-844f-60b9c4d537a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:22.867638 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed951b2-48e5-4c21-844f-60b9c4d537a0 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:22.869502 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d8a9ef-b043-4fe3-9ce8-6e4bab35fe3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:22.870146 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d8a9ef-b043-4fe3-9ce8-6e4bab35fe3a None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:22.870461 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d8a9ef-b043-4fe3-9ce8-6e4bab35fe3a None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:22.870742 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d8a9ef-b043-4fe3-9ce8-6e4bab35fe3a None admin] PATH_INFO: `/v3/users/swift` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:22.876231 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 363/729] 10.222.0.58 () {56 vars in 1103 bytes} [Thu Aug 25 03:01:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:22.882529 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d8a9ef-b043-4fe3-9ce8-6e4bab35fe3a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:22.882767 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed951b2-48e5-4c21-844f-60b9c4d537a0 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:22.884107 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d8a9ef-b043-4fe3-9ce8-6e4bab35fe3a None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:22.907216 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-78d8a9ef-b043-4fe3-9ce8-6e4bab35fe3a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 25 03:01:22.907216 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:22.907216 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:22.907216 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:22.907216 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:22.907216 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:22.907216 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:22.907216 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:22.907216 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:22.907216 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:22.907216 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:22.907216 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:22.907216 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 25 03:01:22.907216 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 25 03:01:22.907216 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 25 03:01:22.907216 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 25 03:01:22.909542 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:22.909542 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:22.909542 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 25 03:01:22.909542 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 03:01:22.909542 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 25 03:01:22.909542 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 03:01:22.909542 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:22.909542 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:22.909542 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:22.909542 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:22.909542 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:22.909542 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:22.909542 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:22.909542 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:22.909542 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:22.909542 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:22.909542 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:22.909542 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:22.911856 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:22.911856 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:22.911856 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 25 03:01:22.911856 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 25 03:01:22.911856 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 25 03:01:22.911856 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 25 03:01:22.911856 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 25 03:01:22.911856 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 25 03:01:22.911856 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 25 03:01:22.911856 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:22.911856 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 367/730] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Aug 25 03:01:22 2022] GET /identity/v3/users/swift => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 03:01:22.918782 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d8a9ef-b043-4fe3-9ce8-6e4bab35fe3a None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:22.984430 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:23.014703 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:23.055232 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d8a9ef-b043-4fe3-9ce8-6e4bab35fe3a None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:23.062274 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed951b2-48e5-4c21-844f-60b9c4d537a0 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:23.093412 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afbf4e9-26ad-4db0-85b9-c7044d1f5620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:23.094035 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afbf4e9-26ad-4db0-85b9-c7044d1f5620 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:23.094338 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afbf4e9-26ad-4db0-85b9-c7044d1f5620 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:23.094616 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afbf4e9-26ad-4db0-85b9-c7044d1f5620 None admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:23.095360 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afbf4e9-26ad-4db0-85b9-c7044d1f5620 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:23.095669 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afbf4e9-26ad-4db0-85b9-c7044d1f5620 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:23.097090 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afbf4e9-26ad-4db0-85b9-c7044d1f5620 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:23.100834 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aeb7bcf-bda0-4867-bdea-cbcca2a47e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:23.101474 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeb7bcf-bda0-4867-bdea-cbcca2a47e0b None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:23.101757 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeb7bcf-bda0-4867-bdea-cbcca2a47e0b None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:23.102029 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeb7bcf-bda0-4867-bdea-cbcca2a47e0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:23.114010 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aeb7bcf-bda0-4867-bdea-cbcca2a47e0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:23.115517 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aeb7bcf-bda0-4867-bdea-cbcca2a47e0b None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:23.116025 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 368/731] 10.222.0.58 () {56 vars in 1135 bytes} [Thu Aug 25 03:01:22 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:01:23.117294 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fff4451-fa25-444e-878d-9764a21920f6 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:23.117583 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fff4451-fa25-444e-878d-9764a21920f6 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:23.117862 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fff4451-fa25-444e-878d-9764a21920f6 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:23.119349 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 369/732] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:01:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:01:23.122439 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fff4451-fa25-444e-878d-9764a21920f6 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:23.135472 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-0aeb7bcf-bda0-4867-bdea-cbcca2a47e0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:23.135472 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:23.135472 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:23.135472 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:23.135472 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:23.135472 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:23.135472 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:23.135472 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:23.135472 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:23.135472 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:23.135472 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:23.135472 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:23.135472 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 03:01:23.135472 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 03:01:23.135472 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 03:01:23.135472 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 03:01:23.137558 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:23.137558 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:23.137558 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:23.137558 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:23.137558 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:23.137558 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:23.137558 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:23.137558 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:23.137558 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:23.137558 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:23.137558 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:23.137558 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:23.137558 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:23.137558 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:23.137558 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:23.137558 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:23.137558 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 03:01:23.140231 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 03:01:23.140231 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:23.140231 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:23.146378 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 364/733] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:01:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:23.146378 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27caae2-ffcd-420a-b583-ca85ebf2444c None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:23.146378 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27caae2-ffcd-420a-b583-ca85ebf2444c None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:23.146378 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27caae2-ffcd-420a-b583-ca85ebf2444c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:23.163209 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-a27caae2-ffcd-420a-b583-ca85ebf2444c None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:01:23.164416 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-a27caae2-ffcd-420a-b583-ca85ebf2444c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:01:23.231622 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fff4451-fa25-444e-878d-9764a21920f6 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:23.246618 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 365/734] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:01:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 03:01:23.247772 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27caae2-ffcd-420a-b583-ca85ebf2444c None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:23.249531 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:23.269958 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095108f2-aef3-4b69-96fe-4cec158f429a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:23.270707 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095108f2-aef3-4b69-96fe-4cec158f429a None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:23.270990 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095108f2-aef3-4b69-96fe-4cec158f429a None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:23.271269 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095108f2-aef3-4b69-96fe-4cec158f429a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:23.329616 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:23.395474 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095108f2-aef3-4b69-96fe-4cec158f429a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:23.397019 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095108f2-aef3-4b69-96fe-4cec158f429a None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:23.430124 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27caae2-ffcd-420a-b583-ca85ebf2444c None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:23.474560 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2944847-27e3-4785-8bac-c3391e4e43bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:23.475183 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2944847-27e3-4785-8bac-c3391e4e43bf None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:23.475466 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2944847-27e3-4785-8bac-c3391e4e43bf None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:23.475740 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2944847-27e3-4785-8bac-c3391e4e43bf None admin] PATH_INFO: `/v3/domains` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:23.476570 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2944847-27e3-4785-8bac-c3391e4e43bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:23.476873 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2944847-27e3-4785-8bac-c3391e4e43bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:23.479085 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2944847-27e3-4785-8bac-c3391e4e43bf None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:23.494109 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 366/735] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Aug 25 03:01:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:23.495498 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2ac106-7a12-4db5-b130-05276d4b4c9a None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:23.495787 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2ac106-7a12-4db5-b130-05276d4b4c9a None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:23.496065 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2ac106-7a12-4db5-b130-05276d4b4c9a None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:23.497673 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 367/736] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:01:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:01:23.499785 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2ac106-7a12-4db5-b130-05276d4b4c9a None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:23.530011 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 370/737] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Aug 25 03:01:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 25 03:01:23.531966 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975b1a1e-2edb-4996-9817-047e9be98488 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:23.532618 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975b1a1e-2edb-4996-9817-047e9be98488 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:23.532994 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975b1a1e-2edb-4996-9817-047e9be98488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:23.555954 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-975b1a1e-2edb-4996-9817-047e9be98488 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:01:23.557802 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-975b1a1e-2edb-4996-9817-047e9be98488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:01:23.571662 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:23.622746 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:23.632781 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2ac106-7a12-4db5-b130-05276d4b4c9a None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:23.635702 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 371/738] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:01:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 03:01:23.638710 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975b1a1e-2edb-4996-9817-047e9be98488 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:23.684616 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8ecc1e-9ca1-4067-8d65-4bd73c8d047a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:23.686148 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8ecc1e-9ca1-4067-8d65-4bd73c8d047a None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:23.687031 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8ecc1e-9ca1-4067-8d65-4bd73c8d047a None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:23.687842 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8ecc1e-9ca1-4067-8d65-4bd73c8d047a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:23.696133 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8ecc1e-9ca1-4067-8d65-4bd73c8d047a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:23.697905 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8ecc1e-9ca1-4067-8d65-4bd73c8d047a None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:23.711221 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-2c8ecc1e-9ca1-4067-8d65-4bd73c8d047a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 25 03:01:23.711221 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:23.711221 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:23.711221 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:23.711221 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:23.711221 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:23.711221 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:23.711221 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:23.711221 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:23.711221 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:23.711221 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:23.711221 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:23.711221 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 25 03:01:23.711221 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 25 03:01:23.711221 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 25 03:01:23.711221 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 25 03:01:23.712215 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:23.712215 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:23.712215 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 25 03:01:23.712215 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 03:01:23.712215 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 25 03:01:23.712215 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 03:01:23.712215 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:23.712215 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:23.712215 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:23.712215 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:23.712215 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:23.712215 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:23.712215 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:23.712215 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:23.712215 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:23.712215 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:23.712215 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:23.712215 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:23.713158 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:23.713158 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:23.713158 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 25 03:01:23.713158 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 25 03:01:23.713158 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 25 03:01:23.713158 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 25 03:01:23.713158 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 25 03:01:23.713158 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 25 03:01:23.713158 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 25 03:01:23.713158 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:23.721519 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 368/739] 10.222.0.58 () {56 vars in 1117 bytes} [Thu Aug 25 03:01:23 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:23.722712 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8ecc1e-9ca1-4067-8d65-4bd73c8d047a None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:23.768682 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975b1a1e-2edb-4996-9817-047e9be98488 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:23.811806 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b476b1c8-7fa2-4862-96f5-ee04a5692426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:23.812791 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b476b1c8-7fa2-4862-96f5-ee04a5692426 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:23.813329 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b476b1c8-7fa2-4862-96f5-ee04a5692426 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:23.813807 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b476b1c8-7fa2-4862-96f5-ee04a5692426 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:23.824899 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b476b1c8-7fa2-4862-96f5-ee04a5692426 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:23.826652 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b476b1c8-7fa2-4862-96f5-ee04a5692426 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:23.834632 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:23.845097 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-b476b1c8-7fa2-4862-96f5-ee04a5692426 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:23.845097 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:23.845097 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:23.845097 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:23.845097 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:23.845097 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:23.845097 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:23.845097 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:23.845097 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:23.845097 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:23.845097 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:23.845097 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:23.845097 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 03:01:23.845097 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 03:01:23.845097 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 03:01:23.845097 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 03:01:23.850071 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:23.850071 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:23.850071 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:23.850071 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:23.850071 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:23.850071 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:23.850071 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:23.850071 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:23.850071 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:23.850071 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:23.850071 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:23.850071 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:23.850071 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:23.850071 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:23.850071 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:23.850071 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:23.850071 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 03:01:23.851018 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 03:01:23.851018 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:23.851018 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:23.851018 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 372/740] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:01:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:23.851018 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b476b1c8-7fa2-4862-96f5-ee04a5692426 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:23.851293 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8ecc1e-9ca1-4067-8d65-4bd73c8d047a None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:23.896995 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa322ba-2376-421f-8821-c506aebc219f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:23.897728 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa322ba-2376-421f-8821-c506aebc219f None admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:23.898081 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa322ba-2376-421f-8821-c506aebc219f None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:23.898450 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa322ba-2376-421f-8821-c506aebc219f None admin] PATH_INFO: `/v3/services` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:23.899591 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa322ba-2376-421f-8821-c506aebc219f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:23.901335 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa322ba-2376-421f-8821-c506aebc219f None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:23.960839 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b476b1c8-7fa2-4862-96f5-ee04a5692426 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:23.994188 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca419c80-be38-4fc0-b0a1-5e2b75ac26a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:23.994837 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca419c80-be38-4fc0-b0a1-5e2b75ac26a3 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:23.995119 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca419c80-be38-4fc0-b0a1-5e2b75ac26a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:23.995387 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca419c80-be38-4fc0-b0a1-5e2b75ac26a3 None admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:23.996113 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca419c80-be38-4fc0-b0a1-5e2b75ac26a3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:23.996417 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca419c80-be38-4fc0-b0a1-5e2b75ac26a3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:23.997860 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca419c80-be38-4fc0-b0a1-5e2b75ac26a3 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:24.002321 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 369/741] 10.222.0.58 () {60 vars in 1141 bytes} [Thu Aug 25 03:01:23 2022] POST /identity/v3/services => generated 251 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:01:24.002321 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa322ba-2376-421f-8821-c506aebc219f None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:24.028225 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:24.028442 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 373/742] 10.222.0.58 () {56 vars in 1149 bytes} [Thu Aug 25 03:01:23 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:24.028442 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca419c80-be38-4fc0-b0a1-5e2b75ac26a3 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:24.098441 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:24.140626 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa322ba-2376-421f-8821-c506aebc219f None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:24.150326 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca419c80-be38-4fc0-b0a1-5e2b75ac26a3 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:24.183178 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4a1ebe-5f0f-40e6-961d-81a4a6a162a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:24.183795 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4a1ebe-5f0f-40e6-961d-81a4a6a162a5 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:24.184072 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4a1ebe-5f0f-40e6-961d-81a4a6a162a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:24.184340 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4a1ebe-5f0f-40e6-961d-81a4a6a162a5 None admin] PATH_INFO: `/v3/domains` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:24.185008 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4a1ebe-5f0f-40e6-961d-81a4a6a162a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:24.185290 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4a1ebe-5f0f-40e6-961d-81a4a6a162a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:24.186727 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4a1ebe-5f0f-40e6-961d-81a4a6a162a5 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:24.188711 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8220fd-1752-49f9-be42-4e071a5c34da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:24.191969 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8220fd-1752-49f9-be42-4e071a5c34da None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:24.191969 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8220fd-1752-49f9-be42-4e071a5c34da None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:24.191969 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8220fd-1752-49f9-be42-4e071a5c34da None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:24.198835 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 370/743] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Aug 25 03:01:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:24.200099 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bbd885-c9dc-42d9-ad10-4c103c55ebef None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:24.200384 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bbd885-c9dc-42d9-ad10-4c103c55ebef None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:24.200659 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bbd885-c9dc-42d9-ad10-4c103c55ebef None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:24.206796 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 371/744] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:01:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:01:24.206796 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bbd885-c9dc-42d9-ad10-4c103c55ebef None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:24.209841 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8220fd-1752-49f9-be42-4e071a5c34da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:01:24.214972 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-ba8220fd-1752-49f9-be42-4e071a5c34da None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 03:01:24.214972 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:24.214972 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:24.214972 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:24.214972 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:24.214972 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:24.214972 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:24.214972 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:24.214972 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:24.214972 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:24.214972 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:24.214972 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:24.214972 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:01:24.214972 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:01:24.214972 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:01:24.214972 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:01:24.215872 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:01:24.215872 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:01:24.215872 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:01:24.215872 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:01:24.215872 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:24.215872 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:24.215872 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:24.215872 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:24.215872 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:24.215872 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:24.215872 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:24.215872 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:24.215872 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:24.215872 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:24.215872 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:24.215872 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:24.216694 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:24.216694 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:24.216694 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:24.216694 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:24.216694 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:01:24.216694 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:01:24.216694 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:01:24.216694 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:01:24.216694 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:01:24.216694 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:01:24.216694 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 03:01:24.216694 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:24.218465 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 374/745] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Aug 25 03:01:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:24.219603 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8220fd-1752-49f9-be42-4e071a5c34da None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:24.284530 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:24.318360 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bbd885-c9dc-42d9-ad10-4c103c55ebef None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:24.325954 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8220fd-1752-49f9-be42-4e071a5c34da None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:24.344131 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:24.354538 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b1567f-07ce-4a34-8bf0-10f66a9b30db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:24.355162 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b1567f-07ce-4a34-8bf0-10f66a9b30db None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:24.355446 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b1567f-07ce-4a34-8bf0-10f66a9b30db None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:24.355992 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b1567f-07ce-4a34-8bf0-10f66a9b30db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:24.363630 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00142ba7-b1c6-48e3-9b09-31a9818ffd56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:24.364452 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00142ba7-b1c6-48e3-9b09-31a9818ffd56 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:24.364920 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00142ba7-b1c6-48e3-9b09-31a9818ffd56 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:24.365206 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00142ba7-b1c6-48e3-9b09-31a9818ffd56 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:24.372166 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00142ba7-b1c6-48e3-9b09-31a9818ffd56 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:24.374365 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00142ba7-b1c6-48e3-9b09-31a9818ffd56 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:24.385808 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-00142ba7-b1c6-48e3-9b09-31a9818ffd56 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 03:01:24.385808 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:24.385808 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:24.385808 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:24.385808 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:24.385808 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:24.385808 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:24.385808 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:24.385808 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:24.385808 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:24.385808 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:24.385808 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:24.385808 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 25 03:01:24.385808 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 25 03:01:24.385808 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 25 03:01:24.386756 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 25 03:01:24.386756 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:24.386756 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:24.386756 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:24.386756 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:24.386756 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:24.386756 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:24.386756 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:24.386756 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:24.386756 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:24.386756 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:24.386756 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:24.386756 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:24.386756 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:24.386756 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:24.386756 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:24.386756 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:24.387644 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 25 03:01:24.387644 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 25 03:01:24.387644 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 25 03:01:24.387644 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 25 03:01:24.387644 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 25 03:01:24.387644 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 25 03:01:24.387644 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 03:01:24.387644 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:24.387644 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 375/746] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Aug 25 03:01:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:24.388119 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d9256d-a876-4d39-ad8e-3bafbbbfb8a0 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:24.388755 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d9256d-a876-4d39-ad8e-3bafbbbfb8a0 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:24.388755 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d9256d-a876-4d39-ad8e-3bafbbbfb8a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:24.407266 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-41d9256d-a876-4d39-ad8e-3bafbbbfb8a0 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:01:24.408502 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-41d9256d-a876-4d39-ad8e-3bafbbbfb8a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:01:24.433563 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b1567f-07ce-4a34-8bf0-10f66a9b30db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:24.435108 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b1567f-07ce-4a34-8bf0-10f66a9b30db None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:24.486305 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 376/747] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:01:24 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 03:01:24.486305 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d9256d-a876-4d39-ad8e-3bafbbbfb8a0 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:24.527101 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:24.567184 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 372/748] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Aug 25 03:01:24 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 25 03:01:24.568535 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b1567f-07ce-4a34-8bf0-10f66a9b30db None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:24.597649 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d9256d-a876-4d39-ad8e-3bafbbbfb8a0 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:24.606022 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:24.641347 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da38e61-a938-47f2-9089-ddbf076b3172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:24.642407 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da38e61-a938-47f2-9089-ddbf076b3172 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:24.643218 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da38e61-a938-47f2-9089-ddbf076b3172 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:24.643836 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da38e61-a938-47f2-9089-ddbf076b3172 None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:24.644814 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da38e61-a938-47f2-9089-ddbf076b3172 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:24.645455 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da38e61-a938-47f2-9089-ddbf076b3172 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:24.647651 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da38e61-a938-47f2-9089-ddbf076b3172 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:24.664332 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 377/749] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Aug 25 03:01:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:24.665462 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da38e61-a938-47f2-9089-ddbf076b3172 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:24.690814 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b1567f-07ce-4a34-8bf0-10f66a9b30db None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:24.726199 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db11f951-b8a9-4a4e-96f0-4f35efe6d2a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:24.726919 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db11f951-b8a9-4a4e-96f0-4f35efe6d2a1 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:24.727240 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db11f951-b8a9-4a4e-96f0-4f35efe6d2a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:24.727582 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db11f951-b8a9-4a4e-96f0-4f35efe6d2a1 None admin] PATH_INFO: `/v3/projects` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:24.728341 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db11f951-b8a9-4a4e-96f0-4f35efe6d2a1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:24.728710 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db11f951-b8a9-4a4e-96f0-4f35efe6d2a1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:24.730446 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db11f951-b8a9-4a4e-96f0-4f35efe6d2a1 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:24.747999 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 373/750] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Aug 25 03:01:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:24.747999 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db11f951-b8a9-4a4e-96f0-4f35efe6d2a1 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:24.788362 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:24.794794 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da38e61-a938-47f2-9089-ddbf076b3172 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:24.832550 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bd8139-05d2-4ccb-a47c-08b6a548f604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:24.833394 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bd8139-05d2-4ccb-a47c-08b6a548f604 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:24.833828 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bd8139-05d2-4ccb-a47c-08b6a548f604 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:24.834316 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bd8139-05d2-4ccb-a47c-08b6a548f604 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:24.847041 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bd8139-05d2-4ccb-a47c-08b6a548f604 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:24.848475 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bd8139-05d2-4ccb-a47c-08b6a548f604 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:24.862585 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-97bd8139-05d2-4ccb-a47c-08b6a548f604 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:24.862585 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:24.862585 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:24.862585 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:24.862585 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:24.862585 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:24.862585 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:24.862585 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:24.862585 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:24.862585 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:24.862585 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:24.862585 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:24.862585 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 03:01:24.862585 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 03:01:24.862585 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 03:01:24.862585 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 03:01:24.863713 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:24.863713 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:24.863713 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:24.863713 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:24.863713 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:24.863713 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:24.863713 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:24.863713 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:24.863713 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:24.863713 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:24.863713 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:24.863713 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:24.863713 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:24.863713 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:24.863713 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:24.863713 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:24.863713 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 03:01:24.864835 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 03:01:24.864835 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:24.864835 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:24.865553 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db11f951-b8a9-4a4e-96f0-4f35efe6d2a1 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:24.868764 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 378/751] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:01:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:24.869009 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bd8139-05d2-4ccb-a47c-08b6a548f604 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:24.891886 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:24.904104 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccdd348-bb5a-4a7e-a8e8-94785b596364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:24.904737 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccdd348-bb5a-4a7e-a8e8-94785b596364 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:24.905019 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccdd348-bb5a-4a7e-a8e8-94785b596364 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:24.905327 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccdd348-bb5a-4a7e-a8e8-94785b596364 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:24.916055 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccdd348-bb5a-4a7e-a8e8-94785b596364 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:24.917550 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccdd348-bb5a-4a7e-a8e8-94785b596364 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:24.933378 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-8ccdd348-bb5a-4a7e-a8e8-94785b596364 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:24.933378 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:24.933378 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:24.933378 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:24.933378 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:24.933378 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:24.933378 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:24.933378 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:24.933378 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:24.933378 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:24.933378 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:24.933378 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:24.933378 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 03:01:24.933378 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 03:01:24.933378 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 03:01:24.933378 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 03:01:24.934315 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:24.934315 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:24.934315 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:24.934315 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:24.934315 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:24.934315 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:24.934315 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:24.934315 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:24.934315 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:24.934315 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:24.934315 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:24.934315 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:24.934315 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:24.934315 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:24.934315 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:24.934315 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:24.934315 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 03:01:24.935205 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 03:01:24.935205 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:24.935205 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:24.935205 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 374/752] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:01:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:24.935424 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccdd348-bb5a-4a7e-a8e8-94785b596364 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:24.982044 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bd8139-05d2-4ccb-a47c-08b6a548f604 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:25.022818 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b503f485-4951-4f64-bbf7-3fc6cc008cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:25.023433 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b503f485-4951-4f64-bbf7-3fc6cc008cbf None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:25.023718 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b503f485-4951-4f64-bbf7-3fc6cc008cbf None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:25.023993 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b503f485-4951-4f64-bbf7-3fc6cc008cbf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:25.024745 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b503f485-4951-4f64-bbf7-3fc6cc008cbf None admin] RBAC: Adding query filter params (role.id=62f19ea272b447ccaab1576cd9bf4fd9, scope.project.id=9fd4be43faec4e4191e0e4ec4a4588c3, user.id=5047e1a7cd2d416d90b2d8c5a6596522) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:25.024897 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:25.025034 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b503f485-4951-4f64-bbf7-3fc6cc008cbf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:25.026593 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b503f485-4951-4f64-bbf7-3fc6cc008cbf None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:25.042308 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 379/753] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Aug 25 03:01:24 2022] GET /identity/v3/role_assignments?user.id=5047e1a7cd2d416d90b2d8c5a6596522&scope.project.id=9fd4be43faec4e4191e0e4ec4a4588c3&role.id=62f19ea272b447ccaab1576cd9bf4fd9 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:25.042308 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b503f485-4951-4f64-bbf7-3fc6cc008cbf None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:25.055202 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccdd348-bb5a-4a7e-a8e8-94785b596364 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:25.102949 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f6eacd-da2c-4396-a2c5-926e4fe2186c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:25.103469 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f6eacd-da2c-4396-a2c5-926e4fe2186c None admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:25.103747 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f6eacd-da2c-4396-a2c5-926e4fe2186c None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:25.104020 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f6eacd-da2c-4396-a2c5-926e4fe2186c None admin] PATH_INFO: `/v3/services` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:25.104682 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f6eacd-da2c-4396-a2c5-926e4fe2186c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:25.106102 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f6eacd-da2c-4396-a2c5-926e4fe2186c None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:25.147296 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:25.164773 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b503f485-4951-4f64-bbf7-3fc6cc008cbf None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:25.202473 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84118eab-1e92-4bb2-a480-cd59d74105e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:25.203378 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84118eab-1e92-4bb2-a480-cd59d74105e2 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:25.203658 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84118eab-1e92-4bb2-a480-cd59d74105e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:25.203931 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84118eab-1e92-4bb2-a480-cd59d74105e2 None admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:25.204581 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84118eab-1e92-4bb2-a480-cd59d74105e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:25.204884 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84118eab-1e92-4bb2-a480-cd59d74105e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:25.206330 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84118eab-1e92-4bb2-a480-cd59d74105e2 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:25.208490 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 375/754] 10.222.0.58 () {60 vars in 1141 bytes} [Thu Aug 25 03:01:24 2022] POST /identity/v3/services => generated 240 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:01:25.209901 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f6eacd-da2c-4396-a2c5-926e4fe2186c None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:25.218881 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 380/755] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Aug 25 03:01:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:25.225268 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84118eab-1e92-4bb2-a480-cd59d74105e2 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:25.356818 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f6eacd-da2c-4396-a2c5-926e4fe2186c None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:25.399530 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858adab6-1e11-4d16-86bd-774987dd1e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:25.400781 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858adab6-1e11-4d16-86bd-774987dd1e76 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:25.401662 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858adab6-1e11-4d16-86bd-774987dd1e76 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:25.402444 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858adab6-1e11-4d16-86bd-774987dd1e76 None admin] PATH_INFO: `/v3/domains` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:25.403315 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858adab6-1e11-4d16-86bd-774987dd1e76 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:25.403997 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858adab6-1e11-4d16-86bd-774987dd1e76 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:25.407056 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858adab6-1e11-4d16-86bd-774987dd1e76 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:25.411186 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84118eab-1e92-4bb2-a480-cd59d74105e2 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:25.421661 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:25.425024 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 376/756] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Aug 25 03:01:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:25.431003 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858adab6-1e11-4d16-86bd-774987dd1e76 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:25.450928 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3c0c18-6589-471f-ac8a-7fce5175131e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:25.451531 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3c0c18-6589-471f-ac8a-7fce5175131e None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:25.451924 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3c0c18-6589-471f-ac8a-7fce5175131e None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:25.452891 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3c0c18-6589-471f-ac8a-7fce5175131e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:25.461474 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3c0c18-6589-471f-ac8a-7fce5175131e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:25.462724 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3c0c18-6589-471f-ac8a-7fce5175131e None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:25.474692 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-ca3c0c18-6589-471f-ac8a-7fce5175131e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 03:01:25.474692 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:25.474692 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:25.474692 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:25.474692 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:25.474692 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:25.474692 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:25.474692 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:25.474692 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:25.474692 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:25.474692 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:25.474692 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:25.474692 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 25 03:01:25.474692 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 25 03:01:25.474692 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 25 03:01:25.475708 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 25 03:01:25.475708 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:25.475708 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:25.475708 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:25.475708 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:25.475708 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:25.475708 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:25.475708 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:25.475708 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:25.475708 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:25.475708 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:25.475708 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:25.475708 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:25.475708 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:25.475708 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:25.475708 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:25.475708 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:25.476615 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 25 03:01:25.476615 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 25 03:01:25.476615 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 25 03:01:25.476615 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 25 03:01:25.476615 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 25 03:01:25.476615 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 25 03:01:25.476615 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 03:01:25.476615 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:25.481641 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 381/757] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Aug 25 03:01:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 03:01:25.482114 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3c0c18-6589-471f-ac8a-7fce5175131e None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:25.486721 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:25.565628 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858adab6-1e11-4d16-86bd-774987dd1e76 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:25.595184 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3c0c18-6589-471f-ac8a-7fce5175131e None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:25.611167 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e063f353-07b5-4c90-bcdf-763d3feb0331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:25.612292 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e063f353-07b5-4c90-bcdf-763d3feb0331 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:25.613053 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e063f353-07b5-4c90-bcdf-763d3feb0331 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:25.613935 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e063f353-07b5-4c90-bcdf-763d3feb0331 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:25.624811 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e063f353-07b5-4c90-bcdf-763d3feb0331 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:25.627433 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e063f353-07b5-4c90-bcdf-763d3feb0331 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:25.636281 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbd719d-fb9d-421e-a39a-a6d678826fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:25.636906 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbd719d-fb9d-421e-a39a-a6d678826fb5 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:25.637189 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbd719d-fb9d-421e-a39a-a6d678826fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:25.637465 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbd719d-fb9d-421e-a39a-a6d678826fb5 None admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:25.638169 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbd719d-fb9d-421e-a39a-a6d678826fb5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:25.638525 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbd719d-fb9d-421e-a39a-a6d678826fb5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:25.640493 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbd719d-fb9d-421e-a39a-a6d678826fb5 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:25.642844 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-e063f353-07b5-4c90-bcdf-763d3feb0331 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 25 03:01:25.642844 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:25.642844 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:25.642844 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:25.642844 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:25.642844 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:25.642844 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:25.642844 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:25.642844 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:25.642844 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:25.642844 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:25.642844 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:25.642844 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 25 03:01:25.642844 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 25 03:01:25.642844 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 25 03:01:25.642844 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 25 03:01:25.643978 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:25.643978 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:25.643978 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 25 03:01:25.643978 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 03:01:25.643978 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 25 03:01:25.643978 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 03:01:25.643978 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:25.643978 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:25.643978 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:25.643978 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:25.643978 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:25.643978 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:25.643978 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:25.643978 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:25.643978 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:25.643978 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:25.643978 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:25.643978 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:25.645139 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:25.645139 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:25.645139 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 25 03:01:25.645139 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 25 03:01:25.645139 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 25 03:01:25.645139 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 25 03:01:25.645139 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 25 03:01:25.645139 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 25 03:01:25.645139 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 25 03:01:25.645139 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:25.647842 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 377/758] 10.222.0.58 () {56 vars in 1093 bytes} [Thu Aug 25 03:01:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:25.656067 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e063f353-07b5-4c90-bcdf-763d3feb0331 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:25.669631 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 382/759] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Aug 25 03:01:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:25.677037 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbd719d-fb9d-421e-a39a-a6d678826fb5 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:25.777304 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:25.782672 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:25.842108 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e063f353-07b5-4c90-bcdf-763d3feb0331 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:25.845276 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbd719d-fb9d-421e-a39a-a6d678826fb5 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:25.900106 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4b25aa-ba1e-470c-b22e-5b722a4512b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:25.900637 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4b25aa-ba1e-470c-b22e-5b722a4512b4 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:25.900919 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4b25aa-ba1e-470c-b22e-5b722a4512b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:25.901194 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4b25aa-ba1e-470c-b22e-5b722a4512b4 None admin] PATH_INFO: `/v3/users` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:25.901933 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4b25aa-ba1e-470c-b22e-5b722a4512b4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:25.902239 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4b25aa-ba1e-470c-b22e-5b722a4512b4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:25.905467 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7188acf4-e38b-40b8-b701-357cf8ac58cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:25.906073 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7188acf4-e38b-40b8-b701-357cf8ac58cb None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:25.906376 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7188acf4-e38b-40b8-b701-357cf8ac58cb None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:25.906651 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7188acf4-e38b-40b8-b701-357cf8ac58cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:25.907374 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7188acf4-e38b-40b8-b701-357cf8ac58cb None admin] RBAC: Adding query filter params (role.id=a93ddb1fd6f6459fb46a17573b821422, scope.project.id=9fd4be43faec4e4191e0e4ec4a4588c3, user.id=712c35b765b64919aa3eaaf02892590d) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:25.908274 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4b25aa-ba1e-470c-b22e-5b722a4512b4 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:25.910088 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7188acf4-e38b-40b8-b701-357cf8ac58cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:25.911581 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7188acf4-e38b-40b8-b701-357cf8ac58cb None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:25.932737 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 383/760] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Aug 25 03:01:25 2022] GET /identity/v3/role_assignments?user.id=712c35b765b64919aa3eaaf02892590d&scope.project.id=9fd4be43faec4e4191e0e4ec4a4588c3&role.id=a93ddb1fd6f6459fb46a17573b821422 => generated 259 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:01:25.934438 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 378/761] 10.222.0.58 () {56 vars in 1133 bytes} [Thu Aug 25 03:01:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:01:25.940186 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7188acf4-e38b-40b8-b701-357cf8ac58cb None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:26.054470 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd3c06c-81e4-41c6-b21f-708b8fc09ec7 None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:26.055045 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd3c06c-81e4-41c6-b21f-708b8fc09ec7 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:26.055520 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd3c06c-81e4-41c6-b21f-708b8fc09ec7 None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:26.063154 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 379/762] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:01:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:01:26.066308 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7247d1eb-4dc0-4c73-9d88-9d3f1a80c780 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:26.066834 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7247d1eb-4dc0-4c73-9d88-9d3f1a80c780 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:26.067387 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7247d1eb-4dc0-4c73-9d88-9d3f1a80c780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:26.109620 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-7247d1eb-4dc0-4c73-9d88-9d3f1a80c780 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:01:26.113596 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-7247d1eb-4dc0-4c73-9d88-9d3f1a80c780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:01:26.121969 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7188acf4-e38b-40b8-b701-357cf8ac58cb None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:26.157676 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:26.169104 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c00bbc-2cb9-46bb-81c5-afced953c6a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:26.172589 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c00bbc-2cb9-46bb-81c5-afced953c6a0 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:26.172589 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c00bbc-2cb9-46bb-81c5-afced953c6a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:26.172589 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c00bbc-2cb9-46bb-81c5-afced953c6a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:26.183397 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c00bbc-2cb9-46bb-81c5-afced953c6a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:26.190482 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c00bbc-2cb9-46bb-81c5-afced953c6a0 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:26.207024 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-d7c00bbc-2cb9-46bb-81c5-afced953c6a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:26.207024 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:26.207024 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:26.207024 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:26.207024 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:26.207024 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:26.207024 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:26.207024 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:26.207024 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:26.207024 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:26.207024 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:26.207024 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:26.207024 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 03:01:26.207024 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 03:01:26.207024 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 03:01:26.207024 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 03:01:26.208246 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:26.208246 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:26.208246 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:26.208246 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:26.208246 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:26.208246 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:26.208246 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:26.208246 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:26.208246 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:26.208246 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:26.208246 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:26.208246 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:26.208246 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:26.208246 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:26.208246 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:26.208246 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:26.208246 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 03:01:26.209354 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 03:01:26.209354 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:26.209354 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:26.211174 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 384/763] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:01:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:26.216321 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c00bbc-2cb9-46bb-81c5-afced953c6a0 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:26.266172 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 380/764] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:01:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 03:01:26.329153 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c00bbc-2cb9-46bb-81c5-afced953c6a0 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:26.364989 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01050538-be94-42f6-8f81-2039b5c2f479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:26.365864 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01050538-be94-42f6-8f81-2039b5c2f479 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:26.366312 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01050538-be94-42f6-8f81-2039b5c2f479 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:26.366758 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01050538-be94-42f6-8f81-2039b5c2f479 None admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:26.367621 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01050538-be94-42f6-8f81-2039b5c2f479 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:26.368073 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01050538-be94-42f6-8f81-2039b5c2f479 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:26.369586 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01050538-be94-42f6-8f81-2039b5c2f479 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:26.383149 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 385/765] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Aug 25 03:01:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:26.390312 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7247d1eb-4dc0-4c73-9d88-9d3f1a80c780 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:26.468415 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:26.530063 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7247d1eb-4dc0-4c73-9d88-9d3f1a80c780 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:26.572929 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c86728b-1d00-4e37-a9d8-b95a94f5ce40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:26.573565 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c86728b-1d00-4e37-a9d8-b95a94f5ce40 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:26.573845 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c86728b-1d00-4e37-a9d8-b95a94f5ce40 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:26.574120 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c86728b-1d00-4e37-a9d8-b95a94f5ce40 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:26.594341 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c86728b-1d00-4e37-a9d8-b95a94f5ce40 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:26.595890 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c86728b-1d00-4e37-a9d8-b95a94f5ce40 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:26.609528 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-0c86728b-1d00-4e37-a9d8-b95a94f5ce40 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 03:01:26.609528 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:26.609528 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:26.609528 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:26.609528 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:26.609528 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:26.609528 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:26.609528 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:26.609528 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:26.609528 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:26.609528 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:26.609528 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:26.609528 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 25 03:01:26.609528 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 25 03:01:26.609528 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 25 03:01:26.610633 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 25 03:01:26.610633 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:26.610633 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:26.610633 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:26.610633 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:26.610633 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:26.610633 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:26.610633 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:26.610633 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:26.610633 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:26.610633 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:26.610633 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:26.610633 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:26.610633 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:26.610633 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:26.610633 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:26.610633 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:26.611898 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 25 03:01:26.611898 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 25 03:01:26.611898 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 25 03:01:26.611898 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 25 03:01:26.611898 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 25 03:01:26.611898 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 25 03:01:26.611898 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 03:01:26.611898 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:26.611898 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 381/766] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Aug 25 03:01:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:26.616884 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01050538-be94-42f6-8f81-2039b5c2f479 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:26.683446 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:26.692825 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c86728b-1d00-4e37-a9d8-b95a94f5ce40 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:26.748599 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01050538-be94-42f6-8f81-2039b5c2f479 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:26.786759 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c37ffe9-078f-48e4-8ac4-4746a25b6e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:26.787719 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c37ffe9-078f-48e4-8ac4-4746a25b6e05 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:26.788185 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c37ffe9-078f-48e4-8ac4-4746a25b6e05 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:26.788639 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c37ffe9-078f-48e4-8ac4-4746a25b6e05 None admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:26.789522 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c37ffe9-078f-48e4-8ac4-4746a25b6e05 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:26.790008 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c37ffe9-078f-48e4-8ac4-4746a25b6e05 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:26.791640 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c37ffe9-078f-48e4-8ac4-4746a25b6e05 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:26.805681 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 386/767] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Aug 25 03:01:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:26.815248 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c37ffe9-078f-48e4-8ac4-4746a25b6e05 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:26.911402 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c86728b-1d00-4e37-a9d8-b95a94f5ce40 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:26.922573 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:26.969325 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6111453-00b6-47ad-8585-67d4cff44291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:26.970371 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6111453-00b6-47ad-8585-67d4cff44291 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:26.970873 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6111453-00b6-47ad-8585-67d4cff44291 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:26.971304 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6111453-00b6-47ad-8585-67d4cff44291 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:26.972150 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6111453-00b6-47ad-8585-67d4cff44291 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:26.973774 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6111453-00b6-47ad-8585-67d4cff44291 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:26.976142 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:26.979215 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c37ffe9-078f-48e4-8ac4-4746a25b6e05 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:26.992721 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-a6111453-00b6-47ad-8585-67d4cff44291 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 25 03:01:26.992721 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:26.992721 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 25 03:01:26.992721 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 25 03:01:26.992721 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 25 03:01:26.992721 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 25 03:01:26.992721 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 25 03:01:26.992721 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 03:01:26.992721 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 25 03:01:26.992721 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:26.992721 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 25 03:01:26.992721 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:26.992721 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:26.992721 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:26.992721 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:26.992721 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:26.992721 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:27.003258 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:27.003258 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:27.003258 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:27.003258 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:27.003258 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:27.003258 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:27.003258 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 25 03:01:27.003258 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 25 03:01:27.003258 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 25 03:01:27.003258 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 25 03:01:27.003258 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:27.003258 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:27.003258 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:27.003258 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:27.003258 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:27.003258 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:27.004319 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:27.004319 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:27.004319 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:27.004319 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:27.004319 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:27.004319 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:27.004319 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:27.004319 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:27.004319 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:27.004319 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:27.004319 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 25 03:01:27.004319 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 03:01:27.004319 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 25 03:01:27.004319 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:27.004319 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 382/768] 10.222.0.58 () {56 vars in 1129 bytes} [Thu Aug 25 03:01:26 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:27.005309 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6111453-00b6-47ad-8585-67d4cff44291 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:27.039810 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29d9de0-5259-4abc-abba-d7b84fc824f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:27.040892 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29d9de0-5259-4abc-abba-d7b84fc824f5 None admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:27.042725 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29d9de0-5259-4abc-abba-d7b84fc824f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:27.043172 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29d9de0-5259-4abc-abba-d7b84fc824f5 None admin] PATH_INFO: `/v3/projects/9fd4be43faec4e4191e0e4ec4a4588c3/users/0cd171bf4df94f8fab397e42835feeab/roles/62f19ea272b447ccaab1576cd9bf4fd9` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:27.046670 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29d9de0-5259-4abc-abba-d7b84fc824f5 None admin] RBAC: Authorizing `identity:create_grant(project_id=9fd4be43faec4e4191e0e4ec4a4588c3, user_id=0cd171bf4df94f8fab397e42835feeab, role_id=62f19ea272b447ccaab1576cd9bf4fd9)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:27.049191 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29d9de0-5259-4abc-abba-d7b84fc824f5 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:27.120337 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 387/769] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Aug 25 03:01:26 2022] PUT /identity/v3/projects/9fd4be43faec4e4191e0e4ec4a4588c3/users/0cd171bf4df94f8fab397e42835feeab/roles/62f19ea272b447ccaab1576cd9bf4fd9 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:01:27.140355 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6111453-00b6-47ad-8585-67d4cff44291 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:27.185114 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61f9cc8-5be7-4263-be09-246cf13d53cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:27.185755 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61f9cc8-5be7-4263-be09-246cf13d53cc None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:27.186037 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61f9cc8-5be7-4263-be09-246cf13d53cc None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:27.186334 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61f9cc8-5be7-4263-be09-246cf13d53cc None admin] PATH_INFO: `/v3/services` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:27.187014 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61f9cc8-5be7-4263-be09-246cf13d53cc None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:27.187528 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61f9cc8-5be7-4263-be09-246cf13d53cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:27.189064 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61f9cc8-5be7-4263-be09-246cf13d53cc None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:27.197732 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 383/770] 10.222.0.58 () {56 vars in 1124 bytes} [Thu Aug 25 03:01:26 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:27.204832 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29d9de0-5259-4abc-abba-d7b84fc824f5 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:27.317337 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29d9de0-5259-4abc-abba-d7b84fc824f5 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:27.353022 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6827b106-96f4-4c6d-9cc2-0cf752da1f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:27.353638 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6827b106-96f4-4c6d-9cc2-0cf752da1f93 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:27.353967 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6827b106-96f4-4c6d-9cc2-0cf752da1f93 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:27.354243 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6827b106-96f4-4c6d-9cc2-0cf752da1f93 None admin] PATH_INFO: `/v3/services` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:27.354907 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6827b106-96f4-4c6d-9cc2-0cf752da1f93 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:27.355205 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6827b106-96f4-4c6d-9cc2-0cf752da1f93 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:27.356634 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6827b106-96f4-4c6d-9cc2-0cf752da1f93 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:27.362039 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 388/771] 10.222.0.58 () {56 vars in 1124 bytes} [Thu Aug 25 03:01:27 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:27.367618 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61f9cc8-5be7-4263-be09-246cf13d53cc None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:27.408868 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:27.450901 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d25e85-bcaa-4628-9cab-c6097b719800 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:27.451227 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d25e85-bcaa-4628-9cab-c6097b719800 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:27.451457 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d25e85-bcaa-4628-9cab-c6097b719800 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:27.453872 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 389/772] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:01:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:01:27.459469 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b056eea-6b8b-4fdf-95cd-8a4a278c78ac None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:27.459761 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b056eea-6b8b-4fdf-95cd-8a4a278c78ac None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:27.460042 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b056eea-6b8b-4fdf-95cd-8a4a278c78ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:27.481476 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-9b056eea-6b8b-4fdf-95cd-8a4a278c78ac None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:01:27.482709 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-9b056eea-6b8b-4fdf-95cd-8a4a278c78ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:01:27.500206 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61f9cc8-5be7-4263-be09-246cf13d53cc None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:27.506192 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:27.566726 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee9a1ec-5c70-4e85-99ce-e5467ca37561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:27.567401 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee9a1ec-5c70-4e85-99ce-e5467ca37561 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:27.567626 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee9a1ec-5c70-4e85-99ce-e5467ca37561 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:27.567901 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee9a1ec-5c70-4e85-99ce-e5467ca37561 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:27.568618 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee9a1ec-5c70-4e85-99ce-e5467ca37561 None admin] RBAC: Adding query filter params (interface=public, service_id=07c4074fa37542e9a7561870fe5ae7c2, region_id=RegionOne) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:27.568927 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee9a1ec-5c70-4e85-99ce-e5467ca37561 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:27.571819 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee9a1ec-5c70-4e85-99ce-e5467ca37561 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:27.585631 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 384/773] 10.222.0.58 () {56 vars in 1251 bytes} [Thu Aug 25 03:01:27 2022] GET /identity/v3/endpoints?service_id=07c4074fa37542e9a7561870fe5ae7c2&interface=public®ion_id=RegionOne => generated 194 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:01:27.591118 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee9a1ec-5c70-4e85-99ce-e5467ca37561 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:27.605445 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 390/774] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:01:27 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 25 03:01:27.727842 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:27.734631 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee9a1ec-5c70-4e85-99ce-e5467ca37561 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:27.753208 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510d9e23-224a-4128-b9bb-332a196340dd None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:27.753716 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510d9e23-224a-4128-b9bb-332a196340dd None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:27.753828 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510d9e23-224a-4128-b9bb-332a196340dd None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:27.755348 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 391/775] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:01:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:01:27.766897 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67b47b6-f4cb-470e-8757-74bd34c1eba7 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:27.767201 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67b47b6-f4cb-470e-8757-74bd34c1eba7 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:27.767440 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67b47b6-f4cb-470e-8757-74bd34c1eba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:27.781664 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91702147-329f-41f4-a2f3-0f1168112eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:27.782634 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91702147-329f-41f4-a2f3-0f1168112eeb None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:27.783115 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91702147-329f-41f4-a2f3-0f1168112eeb None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:27.783586 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91702147-329f-41f4-a2f3-0f1168112eeb None admin] PATH_INFO: `/v3/services` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:27.784390 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91702147-329f-41f4-a2f3-0f1168112eeb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:27.785964 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91702147-329f-41f4-a2f3-0f1168112eeb None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:27.791657 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 385/776] 10.222.0.58 () {56 vars in 1087 bytes} [Thu Aug 25 03:01:27 2022] GET /identity/v3/services => generated 772 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:27.806902 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-f67b47b6-f4cb-470e-8757-74bd34c1eba7 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:01:27.808124 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-f67b47b6-f4cb-470e-8757-74bd34c1eba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:01:27.924210 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 392/777] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:01:27 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 03:01:28.034103 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91702147-329f-41f4-a2f3-0f1168112eeb None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:28.139313 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91702147-329f-41f4-a2f3-0f1168112eeb None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:28.188242 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f9fc31-a6c0-47fd-9b1b-25a43a686acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:28.188803 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f9fc31-a6c0-47fd-9b1b-25a43a686acc None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:28.189083 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f9fc31-a6c0-47fd-9b1b-25a43a686acc None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:28.189356 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f9fc31-a6c0-47fd-9b1b-25a43a686acc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:28.209693 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f9fc31-a6c0-47fd-9b1b-25a43a686acc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:01:28.214395 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-55f9fc31-a6c0-47fd-9b1b-25a43a686acc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 03:01:28.214395 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:28.214395 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:28.214395 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:28.214395 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:28.214395 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:28.214395 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:28.214395 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:28.214395 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:28.214395 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:28.214395 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:28.214395 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:28.214395 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:01:28.214395 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:01:28.214395 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:01:28.214395 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:01:28.215315 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:01:28.215315 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:01:28.215315 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:01:28.215315 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:01:28.215315 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:28.215315 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:28.215315 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:28.215315 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:28.215315 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:28.215315 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:28.215315 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:28.215315 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:28.215315 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:28.215315 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:28.215315 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:28.215315 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:28.216142 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:28.216142 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:28.216142 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:28.216142 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:28.216142 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:01:28.216142 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:01:28.216142 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:01:28.216142 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:01:28.216142 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:01:28.216142 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:01:28.216142 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 03:01:28.216142 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:28.217227 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 386/778] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Aug 25 03:01:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:28.222406 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67b47b6-f4cb-470e-8757-74bd34c1eba7 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:28.231922 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322a112a-33a3-4c80-85db-81619b314577 None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:28.232193 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322a112a-33a3-4c80-85db-81619b314577 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:28.232462 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322a112a-33a3-4c80-85db-81619b314577 None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:28.234767 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 387/779] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:01:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:01:28.235582 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322a112a-33a3-4c80-85db-81619b314577 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:28.277316 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:28.324411 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:28.344374 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67b47b6-f4cb-470e-8757-74bd34c1eba7 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:28.379393 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322a112a-33a3-4c80-85db-81619b314577 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:28.390478 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66edb47-c595-4057-9e5d-d701a74322b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:28.391176 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66edb47-c595-4057-9e5d-d701a74322b3 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:28.391873 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66edb47-c595-4057-9e5d-d701a74322b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:28.392211 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66edb47-c595-4057-9e5d-d701a74322b3 None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:28.393248 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66edb47-c595-4057-9e5d-d701a74322b3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:28.393653 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66edb47-c595-4057-9e5d-d701a74322b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:28.395360 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66edb47-c595-4057-9e5d-d701a74322b3 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:28.407082 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 393/780] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Aug 25 03:01:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:28.408940 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd948a-f730-434a-9d67-b5ddbd53df1b None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:28.409477 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd948a-f730-434a-9d67-b5ddbd53df1b None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:28.409752 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd948a-f730-434a-9d67-b5ddbd53df1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:28.426551 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b4ca07-1e8d-45ee-9f0e-1ff85d48769e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:28.427278 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b4ca07-1e8d-45ee-9f0e-1ff85d48769e None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:28.427628 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b4ca07-1e8d-45ee-9f0e-1ff85d48769e None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:28.428055 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b4ca07-1e8d-45ee-9f0e-1ff85d48769e None admin] PATH_INFO: `/v3/services/network` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:28.428718 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b4ca07-1e8d-45ee-9f0e-1ff85d48769e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:28.428951 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-95dd948a-f730-434a-9d67-b5ddbd53df1b None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:01:28.430152 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-95dd948a-f730-434a-9d67-b5ddbd53df1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:01:28.430264 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b4ca07-1e8d-45ee-9f0e-1ff85d48769e None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:28.441315 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-02b4ca07-1e8d-45ee-9f0e-1ff85d48769e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 25 03:01:28.441315 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:28.441315 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 25 03:01:28.441315 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 25 03:01:28.441315 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 25 03:01:28.441315 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 25 03:01:28.441315 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 25 03:01:28.441315 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 03:01:28.441315 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 25 03:01:28.441315 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:28.441315 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 25 03:01:28.441315 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:28.441315 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:28.441315 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:28.441315 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:28.441315 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:28.441315 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:28.443605 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:28.443605 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:28.443605 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:28.443605 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:28.443605 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:28.443605 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:28.443605 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 25 03:01:28.443605 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 25 03:01:28.443605 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 25 03:01:28.443605 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 25 03:01:28.443605 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:28.443605 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:28.443605 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:28.443605 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:28.443605 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:28.443605 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:28.445580 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:28.445580 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:28.445580 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:28.445580 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:28.445580 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:28.445580 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:28.445580 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:28.445580 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:28.445580 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:28.445580 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:28.445580 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 25 03:01:28.445580 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 03:01:28.445580 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 25 03:01:28.445580 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:28.445580 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 388/781] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Aug 25 03:01:28 2022] GET /identity/v3/services/network => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:28.446682 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b4ca07-1e8d-45ee-9f0e-1ff85d48769e None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:28.512424 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 394/782] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:01:28 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 03:01:28.513498 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dd948a-f730-434a-9d67-b5ddbd53df1b None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:28.582703 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:28.619095 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:28.621658 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b4ca07-1e8d-45ee-9f0e-1ff85d48769e None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:28.648708 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dd948a-f730-434a-9d67-b5ddbd53df1b None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:28.684433 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a17a2d0-1669-4b99-8b61-ead29963400e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:28.685362 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a17a2d0-1669-4b99-8b61-ead29963400e None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:28.685833 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a17a2d0-1669-4b99-8b61-ead29963400e None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:28.686307 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a17a2d0-1669-4b99-8b61-ead29963400e None admin] PATH_INFO: `/v3/services` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:28.687137 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a17a2d0-1669-4b99-8b61-ead29963400e None admin] RBAC: Adding query filter params (name=network) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:28.687614 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a17a2d0-1669-4b99-8b61-ead29963400e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:28.689157 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a17a2d0-1669-4b99-8b61-ead29963400e None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:28.694217 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 395/783] 10.222.0.58 () {56 vars in 1112 bytes} [Thu Aug 25 03:01:28 2022] GET /identity/v3/services?name=network => generated 124 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:28.699769 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a17a2d0-1669-4b99-8b61-ead29963400e None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:28.711821 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ac0c31-e3c5-4b13-a619-8ade12a72629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:28.712540 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ac0c31-e3c5-4b13-a619-8ade12a72629 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:28.712894 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ac0c31-e3c5-4b13-a619-8ade12a72629 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:28.713242 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ac0c31-e3c5-4b13-a619-8ade12a72629 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:28.743347 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ac0c31-e3c5-4b13-a619-8ade12a72629 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:28.744914 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ac0c31-e3c5-4b13-a619-8ade12a72629 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:28.772373 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-37ac0c31-e3c5-4b13-a619-8ade12a72629 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:28.772373 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:28.772373 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:28.772373 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:28.772373 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:28.772373 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:28.772373 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:28.772373 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:28.772373 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:28.772373 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:28.772373 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:28.772373 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:28.772373 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 03:01:28.772373 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 03:01:28.772373 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 03:01:28.772373 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 03:01:28.773369 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:28.773369 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:28.773369 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:28.773369 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:28.773369 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:28.773369 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:28.773369 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:28.773369 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:28.773369 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:28.773369 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:28.773369 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:28.773369 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:28.773369 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:28.773369 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:28.773369 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:28.773369 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:28.773369 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 03:01:28.774299 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 03:01:28.774299 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:28.774299 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:28.778166 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 389/784] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:01:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:28.783249 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ac0c31-e3c5-4b13-a619-8ade12a72629 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:28.807757 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a17a2d0-1669-4b99-8b61-ead29963400e None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:28.848995 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03d0a95-63de-44b5-8aee-851da2f52a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:28.849648 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03d0a95-63de-44b5-8aee-851da2f52a0c None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:28.850008 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03d0a95-63de-44b5-8aee-851da2f52a0c None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:28.850305 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03d0a95-63de-44b5-8aee-851da2f52a0c None admin] PATH_INFO: `/v3/services` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:28.850995 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03d0a95-63de-44b5-8aee-851da2f52a0c None admin] RBAC: Adding query filter params (type=network) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:28.851536 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03d0a95-63de-44b5-8aee-851da2f52a0c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:28.852685 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03d0a95-63de-44b5-8aee-851da2f52a0c None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:28.858063 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 396/785] 10.222.0.58 () {56 vars in 1112 bytes} [Thu Aug 25 03:01:28 2022] GET /identity/v3/services?type=network => generated 350 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:01:28.863256 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03d0a95-63de-44b5-8aee-851da2f52a0c None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:28.864884 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:28.925019 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ac0c31-e3c5-4b13-a619-8ade12a72629 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:28.967558 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903f0dd0-f561-4135-b5fd-5327acde0f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:28.967558 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903f0dd0-f561-4135-b5fd-5327acde0f3d None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:28.967558 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903f0dd0-f561-4135-b5fd-5327acde0f3d None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:28.967897 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03d0a95-63de-44b5-8aee-851da2f52a0c None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:28.967999 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903f0dd0-f561-4135-b5fd-5327acde0f3d None admin] PATH_INFO: `/v3/domains` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:28.967999 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903f0dd0-f561-4135-b5fd-5327acde0f3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:28.967999 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903f0dd0-f561-4135-b5fd-5327acde0f3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:28.968777 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903f0dd0-f561-4135-b5fd-5327acde0f3d None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:28.979921 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:28.990027 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 390/786] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Aug 25 03:01:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:28.990027 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903f0dd0-f561-4135-b5fd-5327acde0f3d None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:29.007891 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2925b110-62c8-4d85-b35b-fac260997e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:29.008830 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2925b110-62c8-4d85-b35b-fac260997e80 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:29.009334 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2925b110-62c8-4d85-b35b-fac260997e80 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:29.009844 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2925b110-62c8-4d85-b35b-fac260997e80 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:29.022035 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2925b110-62c8-4d85-b35b-fac260997e80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:01:29.027743 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-2925b110-62c8-4d85-b35b-fac260997e80 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 25 03:01:29.027743 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:29.027743 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:29.027743 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:29.027743 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:29.027743 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:29.027743 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:29.027743 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:29.027743 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:29.027743 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:29.027743 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:29.027743 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:29.027743 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:01:29.027743 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:01:29.027743 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:01:29.027743 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:01:29.028654 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:01:29.028654 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:01:29.028654 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:01:29.028654 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:01:29.028654 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:29.028654 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:29.028654 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:29.028654 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:29.028654 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:29.028654 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:29.028654 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:29.028654 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:29.028654 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:29.028654 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:29.028654 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:29.028654 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:29.029507 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:29.029507 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:29.029507 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:29.029507 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:29.029507 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:01:29.029507 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:01:29.029507 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:01:29.029507 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:01:29.029507 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:01:29.029507 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:01:29.029507 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 25 03:01:29.029507 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:29.032322 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 397/787] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Aug 25 03:01:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:29.033581 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2925b110-62c8-4d85-b35b-fac260997e80 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:29.101079 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:29.122321 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903f0dd0-f561-4135-b5fd-5327acde0f3d None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:29.145495 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2925b110-62c8-4d85-b35b-fac260997e80 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:29.161633 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b870b6e-a7e7-45b5-87e7-69969ae46d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:29.162910 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b870b6e-a7e7-45b5-87e7-69969ae46d7b None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:29.163903 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b870b6e-a7e7-45b5-87e7-69969ae46d7b None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:29.164388 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b870b6e-a7e7-45b5-87e7-69969ae46d7b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:29.165660 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b870b6e-a7e7-45b5-87e7-69969ae46d7b None admin] RBAC: Adding query filter params (interface=public, service_id=8ccf080be3f540df83fe175b82b19b2a, region_id=RegionOne) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:29.166461 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b870b6e-a7e7-45b5-87e7-69969ae46d7b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:29.168438 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b870b6e-a7e7-45b5-87e7-69969ae46d7b None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:29.177871 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 391/788] 10.222.0.58 () {56 vars in 1251 bytes} [Thu Aug 25 03:01:28 2022] GET /identity/v3/endpoints?service_id=8ccf080be3f540df83fe175b82b19b2a&interface=public®ion_id=RegionOne => generated 194 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:29.179133 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b870b6e-a7e7-45b5-87e7-69969ae46d7b None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:29.195793 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051be2e7-2a45-4ed3-9490-2e59d8b6f96f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:29.196495 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051be2e7-2a45-4ed3-9490-2e59d8b6f96f None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:29.196774 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051be2e7-2a45-4ed3-9490-2e59d8b6f96f None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:29.197049 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051be2e7-2a45-4ed3-9490-2e59d8b6f96f None admin] PATH_INFO: `/v3/users/swift` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:29.207986 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051be2e7-2a45-4ed3-9490-2e59d8b6f96f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:29.209687 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051be2e7-2a45-4ed3-9490-2e59d8b6f96f None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:29.225309 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-051be2e7-2a45-4ed3-9490-2e59d8b6f96f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 25 03:01:29.225309 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:29.225309 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:29.225309 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:29.225309 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:29.225309 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:29.225309 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:29.225309 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:29.225309 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:29.225309 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:29.225309 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:29.225309 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:29.225309 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 25 03:01:29.225309 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 25 03:01:29.225309 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 25 03:01:29.225309 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 25 03:01:29.226270 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:29.226270 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:29.226270 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 25 03:01:29.226270 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 03:01:29.226270 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 25 03:01:29.226270 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 03:01:29.226270 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:29.226270 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:29.226270 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:29.226270 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:29.226270 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:29.226270 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:29.226270 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:29.226270 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:29.226270 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:29.226270 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:29.226270 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:29.226270 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:29.227811 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:29.227811 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:29.227811 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 25 03:01:29.227811 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 25 03:01:29.227811 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 25 03:01:29.227811 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 25 03:01:29.227811 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 25 03:01:29.227811 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 25 03:01:29.227811 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 25 03:01:29.227811 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:29.230358 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 398/789] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Aug 25 03:01:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:29.231560 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051be2e7-2a45-4ed3-9490-2e59d8b6f96f None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:29.260010 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:29.311131 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b870b6e-a7e7-45b5-87e7-69969ae46d7b None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:29.355057 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6942809-9105-4a19-99a6-50d2ee1c4990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:29.355816 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6942809-9105-4a19-99a6-50d2ee1c4990 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:29.356178 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6942809-9105-4a19-99a6-50d2ee1c4990 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:29.356569 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6942809-9105-4a19-99a6-50d2ee1c4990 None admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:29.357338 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6942809-9105-4a19-99a6-50d2ee1c4990 None admin] RBAC: Adding query filter params (name=service) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:29.357824 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6942809-9105-4a19-99a6-50d2ee1c4990 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:29.359774 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6942809-9105-4a19-99a6-50d2ee1c4990 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:29.368546 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 392/790] 10.222.0.58 () {56 vars in 1103 bytes} [Thu Aug 25 03:01:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:29.369042 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051be2e7-2a45-4ed3-9490-2e59d8b6f96f None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:29.369689 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6942809-9105-4a19-99a6-50d2ee1c4990 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:29.395006 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:29.407293 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcf986b-5822-49b5-a41e-132d01de5ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:29.408109 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcf986b-5822-49b5-a41e-132d01de5ad6 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:29.408551 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcf986b-5822-49b5-a41e-132d01de5ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:29.408974 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcf986b-5822-49b5-a41e-132d01de5ad6 None admin] PATH_INFO: `/v3/services` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:29.409730 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcf986b-5822-49b5-a41e-132d01de5ad6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:29.411816 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcf986b-5822-49b5-a41e-132d01de5ad6 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:29.419077 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 399/791] 10.222.0.58 () {56 vars in 1087 bytes} [Thu Aug 25 03:01:29 2022] GET /identity/v3/services => generated 772 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:29.420051 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcf986b-5822-49b5-a41e-132d01de5ad6 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:29.497795 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6942809-9105-4a19-99a6-50d2ee1c4990 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:29.537473 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:29.543736 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcf986b-5822-49b5-a41e-132d01de5ad6 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:29.549693 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa875949-170e-4e64-8cc6-b920d871d509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:29.550562 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa875949-170e-4e64-8cc6-b920d871d509 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:29.550994 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa875949-170e-4e64-8cc6-b920d871d509 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:29.551415 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa875949-170e-4e64-8cc6-b920d871d509 None admin] PATH_INFO: `/v3/users` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:29.552314 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa875949-170e-4e64-8cc6-b920d871d509 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:29.552806 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa875949-170e-4e64-8cc6-b920d871d509 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:29.554728 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa875949-170e-4e64-8cc6-b920d871d509 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:29.576352 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 393/792] 10.222.0.58 () {56 vars in 1135 bytes} [Thu Aug 25 03:01:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:29.578143 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b357fea-d207-4b34-9114-5f6b26567b1d None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:29.578600 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b357fea-d207-4b34-9114-5f6b26567b1d None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:29.579156 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b357fea-d207-4b34-9114-5f6b26567b1d None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:29.580863 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 394/793] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:01:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:01:29.583307 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b357fea-d207-4b34-9114-5f6b26567b1d None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:29.587973 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e935d6-7f51-4077-b904-4209684f089b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:29.588910 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e935d6-7f51-4077-b904-4209684f089b None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:29.589416 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e935d6-7f51-4077-b904-4209684f089b None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:29.589887 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e935d6-7f51-4077-b904-4209684f089b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:29.603883 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e935d6-7f51-4077-b904-4209684f089b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:29.604711 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e935d6-7f51-4077-b904-4209684f089b None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:29.621747 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-a6e935d6-7f51-4077-b904-4209684f089b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:29.621747 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:29.621747 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:29.621747 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:29.621747 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:29.621747 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:29.621747 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:29.621747 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:29.621747 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:29.621747 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:29.621747 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:29.621747 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:29.621747 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 03:01:29.621747 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 03:01:29.621747 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 03:01:29.621747 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 03:01:29.622793 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:29.622793 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:29.622793 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:29.622793 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:29.622793 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:29.622793 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:29.622793 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:29.622793 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:29.622793 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:29.622793 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:29.622793 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:29.622793 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:29.622793 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:29.622793 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:29.622793 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:29.622793 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:29.622793 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 03:01:29.625134 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 03:01:29.625134 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:29.625134 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:29.625134 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 400/794] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:01:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:29.625862 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b149049-33a7-4e80-851a-1d1481db383e None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:29.626139 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b149049-33a7-4e80-851a-1d1481db383e None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:29.626487 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b149049-33a7-4e80-851a-1d1481db383e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:29.644702 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-1b149049-33a7-4e80-851a-1d1481db383e None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:01:29.645888 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-1b149049-33a7-4e80-851a-1d1481db383e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:01:29.682522 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:29.700145 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b357fea-d207-4b34-9114-5f6b26567b1d None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:29.726392 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 401/795] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:01:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 03:01:29.727481 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b149049-33a7-4e80-851a-1d1481db383e None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:29.747088 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26825830-576b-4145-839f-24fd064fb5dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:29.747737 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26825830-576b-4145-839f-24fd064fb5dc None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:29.748074 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26825830-576b-4145-839f-24fd064fb5dc None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:29.749181 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26825830-576b-4145-839f-24fd064fb5dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:29.767744 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26825830-576b-4145-839f-24fd064fb5dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:29.769207 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26825830-576b-4145-839f-24fd064fb5dc None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:29.795661 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-26825830-576b-4145-839f-24fd064fb5dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:29.795661 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:29.795661 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:29.795661 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:29.795661 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:29.795661 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:29.795661 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:29.795661 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:29.795661 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:29.795661 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:29.795661 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:29.795661 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:29.795661 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 03:01:29.795661 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 03:01:29.795661 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 03:01:29.795661 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 03:01:29.797193 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:29.797193 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:29.797193 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:29.797193 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:29.797193 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:29.797193 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:29.797193 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:29.797193 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:29.797193 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:29.797193 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:29.797193 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:29.797193 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:29.797193 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:29.797193 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:29.797193 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:29.797193 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:29.797193 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 03:01:29.798068 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 03:01:29.798068 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:29.798068 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:29.799495 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 395/796] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:01:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 03:01:29.805320 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26825830-576b-4145-839f-24fd064fb5dc None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:29.876879 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:29.897713 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b149049-33a7-4e80-851a-1d1481db383e None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:29.937064 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895ead64-c9b4-4eec-9843-2ced6be3de59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:29.937678 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895ead64-c9b4-4eec-9843-2ced6be3de59 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:29.937959 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895ead64-c9b4-4eec-9843-2ced6be3de59 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:29.938233 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895ead64-c9b4-4eec-9843-2ced6be3de59 None admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:29.939113 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895ead64-c9b4-4eec-9843-2ced6be3de59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:29.939419 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895ead64-c9b4-4eec-9843-2ced6be3de59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:29.940789 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895ead64-c9b4-4eec-9843-2ced6be3de59 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:29.965848 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 402/797] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Aug 25 03:01:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:29.972113 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895ead64-c9b4-4eec-9843-2ced6be3de59 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:30.012623 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26825830-576b-4145-839f-24fd064fb5dc None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:30.013583 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:30.072283 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb1b208-778f-417b-b0d3-f7deffb37a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:30.072918 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb1b208-778f-417b-b0d3-f7deffb37a1f None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:30.073547 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb1b208-778f-417b-b0d3-f7deffb37a1f None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:30.073821 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb1b208-778f-417b-b0d3-f7deffb37a1f None admin] PATH_INFO: `/v3/domains` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:30.074790 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb1b208-778f-417b-b0d3-f7deffb37a1f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:30.075106 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb1b208-778f-417b-b0d3-f7deffb37a1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:30.076825 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb1b208-778f-417b-b0d3-f7deffb37a1f None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:30.093384 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 396/798] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Aug 25 03:01:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:30.094715 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb1b208-778f-417b-b0d3-f7deffb37a1f None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:30.101247 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895ead64-c9b4-4eec-9843-2ced6be3de59 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:30.144942 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e68ba3-514a-49b7-b203-ea94b6b27d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:30.146041 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e68ba3-514a-49b7-b203-ea94b6b27d41 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:30.146344 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e68ba3-514a-49b7-b203-ea94b6b27d41 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:30.146622 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e68ba3-514a-49b7-b203-ea94b6b27d41 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:30.156960 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e68ba3-514a-49b7-b203-ea94b6b27d41 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:30.158978 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e68ba3-514a-49b7-b203-ea94b6b27d41 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:30.173677 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-39e68ba3-514a-49b7-b203-ea94b6b27d41 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 25 03:01:30.173677 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:30.173677 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:30.173677 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:30.173677 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:30.173677 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:30.173677 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:30.173677 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:30.173677 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:30.173677 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:30.173677 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:30.173677 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:30.173677 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 25 03:01:30.173677 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 25 03:01:30.173677 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 25 03:01:30.173677 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 25 03:01:30.174690 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:30.174690 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:30.174690 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 25 03:01:30.174690 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 03:01:30.174690 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 25 03:01:30.174690 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 03:01:30.174690 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:30.174690 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:30.174690 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:30.174690 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:30.174690 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:30.174690 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:30.174690 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:30.174690 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:30.174690 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:30.174690 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:30.174690 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:30.174690 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:30.175608 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:30.175608 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:30.175608 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 25 03:01:30.175608 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 25 03:01:30.175608 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 25 03:01:30.175608 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 25 03:01:30.175608 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 25 03:01:30.175608 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 25 03:01:30.175608 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 25 03:01:30.175608 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:30.175608 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 403/799] 10.222.0.58 () {56 vars in 1117 bytes} [Thu Aug 25 03:01:29 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:30.176207 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e68ba3-514a-49b7-b203-ea94b6b27d41 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:30.213283 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:30.219514 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb1b208-778f-417b-b0d3-f7deffb37a1f None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:30.253311 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e56d7ec-57c2-496c-89c0-2c843f4387ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:30.253938 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e56d7ec-57c2-496c-89c0-2c843f4387ba None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:30.254216 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e56d7ec-57c2-496c-89c0-2c843f4387ba None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:30.254530 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e56d7ec-57c2-496c-89c0-2c843f4387ba None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:30.265563 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e56d7ec-57c2-496c-89c0-2c843f4387ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:01:30.270165 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-2e56d7ec-57c2-496c-89c0-2c843f4387ba None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 03:01:30.270165 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:30.270165 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:30.270165 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:30.270165 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:30.270165 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:30.270165 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:30.270165 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:30.270165 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:30.270165 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:30.270165 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:30.270165 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:30.270165 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:01:30.270165 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:01:30.270165 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:01:30.270165 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:01:30.271284 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:01:30.271284 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:01:30.271284 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:01:30.271284 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:01:30.271284 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:30.271284 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:30.271284 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:30.271284 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:30.271284 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:30.271284 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:30.271284 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:30.271284 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:30.271284 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:30.271284 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:30.271284 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:30.271284 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:30.272307 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:30.272307 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:30.272307 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:30.272307 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:30.272307 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:01:30.272307 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:01:30.272307 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:01:30.272307 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:01:30.272307 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:01:30.272307 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:01:30.272307 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 03:01:30.272307 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:30.275937 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 397/800] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Aug 25 03:01:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:30.276758 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e56d7ec-57c2-496c-89c0-2c843f4387ba None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:30.283305 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:30.296539 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e68ba3-514a-49b7-b203-ea94b6b27d41 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:30.334769 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0039dcbd-acd4-4503-8114-4ac2f9a10dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:30.335397 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0039dcbd-acd4-4503-8114-4ac2f9a10dfc None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:30.335677 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0039dcbd-acd4-4503-8114-4ac2f9a10dfc None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:30.335948 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0039dcbd-acd4-4503-8114-4ac2f9a10dfc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:30.343188 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0039dcbd-acd4-4503-8114-4ac2f9a10dfc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:30.344615 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0039dcbd-acd4-4503-8114-4ac2f9a10dfc None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:30.355587 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-0039dcbd-acd4-4503-8114-4ac2f9a10dfc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 03:01:30.355587 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:30.355587 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:30.355587 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:30.355587 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:30.355587 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:30.355587 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:30.355587 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:30.355587 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:30.355587 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:30.355587 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:30.355587 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:30.355587 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 25 03:01:30.355587 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 25 03:01:30.355587 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 25 03:01:30.356517 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 25 03:01:30.356517 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:30.356517 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:30.356517 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:30.356517 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:30.356517 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:30.356517 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:30.356517 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:30.356517 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:30.356517 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:30.356517 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:30.356517 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:30.356517 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:30.356517 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:30.356517 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:30.356517 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:30.356517 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:30.357404 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 25 03:01:30.357404 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 25 03:01:30.357404 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 25 03:01:30.357404 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 25 03:01:30.357404 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 25 03:01:30.357404 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 25 03:01:30.357404 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 03:01:30.357404 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:30.357404 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 404/801] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Aug 25 03:01:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:30.358336 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed95de46-7cad-42dd-827d-d98fc7ca5698 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:30.358336 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed95de46-7cad-42dd-827d-d98fc7ca5698 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:30.358336 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed95de46-7cad-42dd-827d-d98fc7ca5698 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:30.363817 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 405/802] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:01:30 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:01:30.363817 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed95de46-7cad-42dd-827d-d98fc7ca5698 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:30.394813 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e56d7ec-57c2-496c-89c0-2c843f4387ba None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:30.430348 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8551e3f0-b704-47e5-8453-500dd0f2c943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:30.430958 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8551e3f0-b704-47e5-8453-500dd0f2c943 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:30.431239 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8551e3f0-b704-47e5-8453-500dd0f2c943 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:30.431513 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8551e3f0-b704-47e5-8453-500dd0f2c943 None admin] PATH_INFO: `/v3/users` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:30.432236 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8551e3f0-b704-47e5-8453-500dd0f2c943 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:30.432546 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8551e3f0-b704-47e5-8453-500dd0f2c943 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:30.433983 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8551e3f0-b704-47e5-8453-500dd0f2c943 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:30.452960 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 398/803] 10.222.0.58 () {56 vars in 1149 bytes} [Thu Aug 25 03:01:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:30.453791 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8551e3f0-b704-47e5-8453-500dd0f2c943 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:30.469920 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed95de46-7cad-42dd-827d-d98fc7ca5698 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:30.471259 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:30.507795 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61715a9b-de33-44a2-aa87-14bc10d19920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:30.508454 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61715a9b-de33-44a2-aa87-14bc10d19920 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:30.508735 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61715a9b-de33-44a2-aa87-14bc10d19920 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:30.509009 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61715a9b-de33-44a2-aa87-14bc10d19920 None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:30.509749 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61715a9b-de33-44a2-aa87-14bc10d19920 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:30.510024 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61715a9b-de33-44a2-aa87-14bc10d19920 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:30.511344 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61715a9b-de33-44a2-aa87-14bc10d19920 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:30.520788 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 406/804] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Aug 25 03:01:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:30.521698 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae2936a-445d-4df7-909d-b83ce3842357 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:30.521987 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae2936a-445d-4df7-909d-b83ce3842357 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:30.522262 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae2936a-445d-4df7-909d-b83ce3842357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:30.542009 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-9ae2936a-445d-4df7-909d-b83ce3842357 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:01:30.543454 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-9ae2936a-445d-4df7-909d-b83ce3842357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:01:30.551252 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:30.562920 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8551e3f0-b704-47e5-8453-500dd0f2c943 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:30.615666 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5279b956-537f-449f-80fb-808277c5b812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:30.616277 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5279b956-537f-449f-80fb-808277c5b812 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:30.616558 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5279b956-537f-449f-80fb-808277c5b812 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:30.616834 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5279b956-537f-449f-80fb-808277c5b812 None admin] PATH_INFO: `/v3/projects` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:30.617524 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5279b956-537f-449f-80fb-808277c5b812 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:30.617827 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5279b956-537f-449f-80fb-808277c5b812 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:30.619362 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5279b956-537f-449f-80fb-808277c5b812 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:30.634338 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 407/805] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:01:30 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 03:01:30.634338 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae2936a-445d-4df7-909d-b83ce3842357 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:30.642088 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 399/806] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Aug 25 03:01:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:30.643006 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5279b956-537f-449f-80fb-808277c5b812 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:30.737907 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae2936a-445d-4df7-909d-b83ce3842357 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:30.742335 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:30.770341 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5279b956-537f-449f-80fb-808277c5b812 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:30.773749 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038fc9f8-f81e-45d5-909b-f50ef3891587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:30.774347 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038fc9f8-f81e-45d5-909b-f50ef3891587 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:30.774633 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038fc9f8-f81e-45d5-909b-f50ef3891587 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:30.774907 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038fc9f8-f81e-45d5-909b-f50ef3891587 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:30.789310 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038fc9f8-f81e-45d5-909b-f50ef3891587 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:30.790812 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038fc9f8-f81e-45d5-909b-f50ef3891587 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:30.792556 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:30.806870 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-038fc9f8-f81e-45d5-909b-f50ef3891587 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:30.806870 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:30.806870 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:30.806870 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:30.806870 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:30.806870 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:30.806870 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:30.806870 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:30.806870 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:30.806870 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:30.806870 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:30.806870 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:30.806870 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 03:01:30.806870 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 03:01:30.806870 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 03:01:30.806870 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 03:01:30.809663 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:30.809663 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:30.809663 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:30.809663 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:30.809663 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:30.809663 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:30.809663 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:30.809663 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:30.809663 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:30.809663 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:30.809663 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:30.809663 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:30.809663 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:30.809663 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:30.809663 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:30.809663 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:30.809663 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 03:01:30.812391 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 03:01:30.812391 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:30.812391 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:30.812391 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 408/807] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:01:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:30.812391 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038fc9f8-f81e-45d5-909b-f50ef3891587 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:30.823661 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e013cc62-4e9c-446f-a71b-8d6e239ceaf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:30.824580 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e013cc62-4e9c-446f-a71b-8d6e239ceaf1 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:30.824860 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e013cc62-4e9c-446f-a71b-8d6e239ceaf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:30.825132 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e013cc62-4e9c-446f-a71b-8d6e239ceaf1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:30.838694 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e013cc62-4e9c-446f-a71b-8d6e239ceaf1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:30.840617 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e013cc62-4e9c-446f-a71b-8d6e239ceaf1 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:30.860364 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-e013cc62-4e9c-446f-a71b-8d6e239ceaf1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:30.860364 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:30.860364 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:30.860364 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:30.860364 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:30.860364 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:30.860364 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:30.860364 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:30.860364 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:30.860364 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:30.860364 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:30.860364 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:30.860364 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 03:01:30.860364 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 03:01:30.860364 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 03:01:30.860364 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 03:01:30.861566 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:30.861566 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:30.861566 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:30.861566 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:30.861566 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:30.861566 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:30.861566 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:30.861566 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:30.861566 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:30.861566 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:30.861566 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:30.861566 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:30.861566 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:30.861566 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:30.861566 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:30.861566 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:30.861566 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 03:01:30.863421 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 03:01:30.863421 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:30.863421 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:30.863421 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 400/808] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:01:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:30.863421 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e013cc62-4e9c-446f-a71b-8d6e239ceaf1 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:30.920475 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038fc9f8-f81e-45d5-909b-f50ef3891587 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:30.961628 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2906008-6e79-470b-81da-0341c0e4ba4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:30.962321 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2906008-6e79-470b-81da-0341c0e4ba4a None admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:30.962618 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2906008-6e79-470b-81da-0341c0e4ba4a None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:30.962895 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2906008-6e79-470b-81da-0341c0e4ba4a None admin] PATH_INFO: `/v3/projects/9fd4be43faec4e4191e0e4ec4a4588c3/users/5047e1a7cd2d416d90b2d8c5a6596522/roles/62f19ea272b447ccaab1576cd9bf4fd9` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:30.964800 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2906008-6e79-470b-81da-0341c0e4ba4a None admin] RBAC: Authorizing `identity:create_grant(project_id=9fd4be43faec4e4191e0e4ec4a4588c3, user_id=5047e1a7cd2d416d90b2d8c5a6596522, role_id=62f19ea272b447ccaab1576cd9bf4fd9)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:30.966403 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2906008-6e79-470b-81da-0341c0e4ba4a None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:30.991974 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e013cc62-4e9c-446f-a71b-8d6e239ceaf1 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:31.017986 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:31.026380 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b61db5b-8d12-4a88-bd5d-1c8e6ac35fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:31.026932 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b61db5b-8d12-4a88-bd5d-1c8e6ac35fcb None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:31.027208 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b61db5b-8d12-4a88-bd5d-1c8e6ac35fcb None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:31.027474 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b61db5b-8d12-4a88-bd5d-1c8e6ac35fcb None admin] PATH_INFO: `/v3/domains` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:31.028111 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b61db5b-8d12-4a88-bd5d-1c8e6ac35fcb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:31.028425 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b61db5b-8d12-4a88-bd5d-1c8e6ac35fcb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:31.029778 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b61db5b-8d12-4a88-bd5d-1c8e6ac35fcb None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:31.041483 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 401/809] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Aug 25 03:01:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:31.042262 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b61db5b-8d12-4a88-bd5d-1c8e6ac35fcb None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:31.057154 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 409/810] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Aug 25 03:01:30 2022] PUT /identity/v3/projects/9fd4be43faec4e4191e0e4ec4a4588c3/users/5047e1a7cd2d416d90b2d8c5a6596522/roles/62f19ea272b447ccaab1576cd9bf4fd9 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:01:31.058189 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2906008-6e79-470b-81da-0341c0e4ba4a None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:31.142001 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:31.172725 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b61db5b-8d12-4a88-bd5d-1c8e6ac35fcb None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:31.201821 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2906008-6e79-470b-81da-0341c0e4ba4a None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:31.225990 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12994419-cf34-4c75-8abb-2577ee4e18db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:31.226888 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12994419-cf34-4c75-8abb-2577ee4e18db None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:31.227308 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12994419-cf34-4c75-8abb-2577ee4e18db None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:31.227745 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12994419-cf34-4c75-8abb-2577ee4e18db None admin] PATH_INFO: `/v3/domains` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:31.228540 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12994419-cf34-4c75-8abb-2577ee4e18db None admin] RBAC: Adding query filter params (name=Default) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:31.228844 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12994419-cf34-4c75-8abb-2577ee4e18db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:31.230212 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12994419-cf34-4c75-8abb-2577ee4e18db None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:31.242858 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 402/811] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Aug 25 03:01:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:31.243680 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12994419-cf34-4c75-8abb-2577ee4e18db None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:31.245278 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade0d84d-6a0f-41bd-b86b-e73eec8a9bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:31.245926 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade0d84d-6a0f-41bd-b86b-e73eec8a9bdb None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:31.246211 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade0d84d-6a0f-41bd-b86b-e73eec8a9bdb None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:31.246526 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade0d84d-6a0f-41bd-b86b-e73eec8a9bdb None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:31.247119 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade0d84d-6a0f-41bd-b86b-e73eec8a9bdb None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:31.248495 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade0d84d-6a0f-41bd-b86b-e73eec8a9bdb None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:31.257746 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-ade0d84d-6a0f-41bd-b86b-e73eec8a9bdb None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 25 03:01:31.257746 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:31.257746 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 25 03:01:31.257746 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 25 03:01:31.257746 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 25 03:01:31.257746 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 25 03:01:31.257746 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 25 03:01:31.257746 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 03:01:31.257746 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 25 03:01:31.257746 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:31.257746 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 25 03:01:31.257746 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:31.257746 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:31.257746 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:31.257746 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:31.257746 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:31.257746 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:31.260071 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:31.260071 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:31.260071 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:31.260071 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:31.260071 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:31.260071 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:31.260071 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 25 03:01:31.260071 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 25 03:01:31.260071 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 25 03:01:31.260071 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 25 03:01:31.260071 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:31.260071 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:31.260071 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:31.260071 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:31.260071 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:31.260071 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:31.261969 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:31.261969 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:31.261969 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:31.261969 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:31.261969 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:31.261969 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:31.261969 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:31.261969 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:31.261969 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:31.261969 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:31.261969 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 25 03:01:31.261969 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 03:01:31.261969 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 25 03:01:31.261969 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:31.261969 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 410/812] 10.222.0.58 () {56 vars in 1129 bytes} [Thu Aug 25 03:01:31 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:31.263860 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade0d84d-6a0f-41bd-b86b-e73eec8a9bdb None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:31.294903 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:31.354661 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12994419-cf34-4c75-8abb-2577ee4e18db None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:31.389145 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3209d0cd-aa54-4ab6-850a-c44f27e5ca81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:31.390759 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3209d0cd-aa54-4ab6-850a-c44f27e5ca81 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:31.391137 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3209d0cd-aa54-4ab6-850a-c44f27e5ca81 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:31.391513 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3209d0cd-aa54-4ab6-850a-c44f27e5ca81 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:31.392775 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade0d84d-6a0f-41bd-b86b-e73eec8a9bdb None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:31.398480 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3209d0cd-aa54-4ab6-850a-c44f27e5ca81 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:31.400239 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3209d0cd-aa54-4ab6-850a-c44f27e5ca81 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:31.412606 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:31.415408 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-3209d0cd-aa54-4ab6-850a-c44f27e5ca81 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 03:01:31.415408 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:31.415408 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:31.415408 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:31.415408 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:31.415408 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:31.415408 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:31.415408 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:31.415408 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:31.415408 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:31.415408 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:31.415408 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:31.415408 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 25 03:01:31.415408 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 25 03:01:31.415408 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 25 03:01:31.416362 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 25 03:01:31.416362 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:31.416362 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:31.416362 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:31.416362 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:31.416362 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:31.416362 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:31.416362 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:31.416362 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:31.416362 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:31.416362 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:31.416362 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:31.416362 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:31.416362 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:31.416362 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:31.416362 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:31.416362 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:31.417584 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 25 03:01:31.417584 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 25 03:01:31.417584 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 25 03:01:31.417584 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 25 03:01:31.417584 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 25 03:01:31.417584 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 25 03:01:31.417584 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 03:01:31.417584 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:31.422365 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 403/813] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Aug 25 03:01:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:31.427881 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3209d0cd-aa54-4ab6-850a-c44f27e5ca81 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:31.438704 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4f1e98-4417-471c-a20c-5b4d1bdb74a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:31.439739 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4f1e98-4417-471c-a20c-5b4d1bdb74a0 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:31.440042 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4f1e98-4417-471c-a20c-5b4d1bdb74a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:31.440321 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4f1e98-4417-471c-a20c-5b4d1bdb74a0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:31.451016 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4f1e98-4417-471c-a20c-5b4d1bdb74a0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:31.453758 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4f1e98-4417-471c-a20c-5b4d1bdb74a0 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:31.468622 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-ef4f1e98-4417-471c-a20c-5b4d1bdb74a0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 25 03:01:31.468622 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:31.468622 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:31.468622 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:31.468622 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:31.468622 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:31.468622 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:31.468622 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:31.468622 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:31.468622 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:31.468622 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:31.468622 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:31.468622 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 25 03:01:31.468622 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 25 03:01:31.468622 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 25 03:01:31.468622 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 25 03:01:31.470132 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:31.470132 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:31.470132 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 25 03:01:31.470132 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 03:01:31.470132 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 25 03:01:31.470132 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 03:01:31.470132 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:31.470132 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:31.470132 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:31.470132 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:31.470132 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:31.470132 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:31.470132 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:31.470132 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:31.470132 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:31.470132 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:31.470132 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:31.470132 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:31.471435 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:31.471435 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:31.471435 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 25 03:01:31.471435 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 25 03:01:31.471435 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 25 03:01:31.471435 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 25 03:01:31.471435 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 25 03:01:31.471435 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 25 03:01:31.471435 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 25 03:01:31.471435 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:31.471435 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 411/814] 10.222.0.58 () {56 vars in 1093 bytes} [Thu Aug 25 03:01:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:31.472652 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4f1e98-4417-471c-a20c-5b4d1bdb74a0 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:31.542141 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3209d0cd-aa54-4ab6-850a-c44f27e5ca81 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:31.550641 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:31.583561 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4f1e98-4417-471c-a20c-5b4d1bdb74a0 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:31.586887 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4ba13f-7a63-40a4-bfd3-09ad4a6268ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:31.588095 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4ba13f-7a63-40a4-bfd3-09ad4a6268ee None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:31.588583 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4ba13f-7a63-40a4-bfd3-09ad4a6268ee None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:31.589140 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4ba13f-7a63-40a4-bfd3-09ad4a6268ee None admin] PATH_INFO: `/v3/services` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:31.589991 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4ba13f-7a63-40a4-bfd3-09ad4a6268ee None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:31.590531 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4ba13f-7a63-40a4-bfd3-09ad4a6268ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:31.593451 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4ba13f-7a63-40a4-bfd3-09ad4a6268ee None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:31.599824 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 404/815] 10.222.0.58 () {56 vars in 1124 bytes} [Thu Aug 25 03:01:31 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:31.601311 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4ba13f-7a63-40a4-bfd3-09ad4a6268ee None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:31.617411 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009edaa2-0abd-4c21-b14a-f02b4495919b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:31.618017 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009edaa2-0abd-4c21-b14a-f02b4495919b None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:31.618310 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009edaa2-0abd-4c21-b14a-f02b4495919b None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:31.618590 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009edaa2-0abd-4c21-b14a-f02b4495919b None admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:31.619278 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009edaa2-0abd-4c21-b14a-f02b4495919b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:31.619581 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009edaa2-0abd-4c21-b14a-f02b4495919b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:31.620995 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009edaa2-0abd-4c21-b14a-f02b4495919b None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:31.643830 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 412/816] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Aug 25 03:01:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:31.643830 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59151f4-8ec8-4e3a-bfc0-20e4e2d0ab1b None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:31.643830 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59151f4-8ec8-4e3a-bfc0-20e4e2d0ab1b None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:31.643830 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59151f4-8ec8-4e3a-bfc0-20e4e2d0ab1b None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:31.643830 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 413/817] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:01:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:01:31.643830 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59151f4-8ec8-4e3a-bfc0-20e4e2d0ab1b None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:31.685649 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:31.725662 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4ba13f-7a63-40a4-bfd3-09ad4a6268ee None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:31.750068 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59151f4-8ec8-4e3a-bfc0-20e4e2d0ab1b None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:31.759537 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e52c7d-b7fc-4e83-ab52-19f465533e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:31.760146 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e52c7d-b7fc-4e83-ab52-19f465533e00 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:31.760423 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e52c7d-b7fc-4e83-ab52-19f465533e00 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:31.760758 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e52c7d-b7fc-4e83-ab52-19f465533e00 None admin] PATH_INFO: `/v3/users` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:31.761496 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e52c7d-b7fc-4e83-ab52-19f465533e00 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:31.761799 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e52c7d-b7fc-4e83-ab52-19f465533e00 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:31.763285 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e52c7d-b7fc-4e83-ab52-19f465533e00 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:31.785785 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 405/818] 10.222.0.58 () {56 vars in 1133 bytes} [Thu Aug 25 03:01:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:31.785785 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e52c7d-b7fc-4e83-ab52-19f465533e00 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:31.790807 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071c04b7-e394-40f0-b61e-aecbe66b2be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:31.791690 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071c04b7-e394-40f0-b61e-aecbe66b2be7 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:31.792221 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071c04b7-e394-40f0-b61e-aecbe66b2be7 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:31.792701 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071c04b7-e394-40f0-b61e-aecbe66b2be7 None admin] PATH_INFO: `/v3/services` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:31.793652 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071c04b7-e394-40f0-b61e-aecbe66b2be7 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:31.794113 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071c04b7-e394-40f0-b61e-aecbe66b2be7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:31.795507 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071c04b7-e394-40f0-b61e-aecbe66b2be7 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:31.805168 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 414/819] 10.222.0.58 () {56 vars in 1124 bytes} [Thu Aug 25 03:01:31 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:31.805168 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4634f0d-08cb-429b-9a1c-dd3b3d4a0241 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:31.805168 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4634f0d-08cb-429b-9a1c-dd3b3d4a0241 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:31.805168 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4634f0d-08cb-429b-9a1c-dd3b3d4a0241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:31.813689 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:31.820318 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-b4634f0d-08cb-429b-9a1c-dd3b3d4a0241 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:01:31.821493 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-b4634f0d-08cb-429b-9a1c-dd3b3d4a0241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:01:31.888755 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e52c7d-b7fc-4e83-ab52-19f465533e00 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:31.888941 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 415/820] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:01:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 03:01:31.890308 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4634f0d-08cb-429b-9a1c-dd3b3d4a0241 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:31.925539 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bcd314-0600-4fe6-9167-b0d0facfc988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:31.926553 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bcd314-0600-4fe6-9167-b0d0facfc988 None admin] REQUEST_METHOD: `PUT` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:31.927039 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bcd314-0600-4fe6-9167-b0d0facfc988 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:31.927466 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bcd314-0600-4fe6-9167-b0d0facfc988 None admin] PATH_INFO: `/v3/projects/9fd4be43faec4e4191e0e4ec4a4588c3/users/712c35b765b64919aa3eaaf02892590d/roles/a93ddb1fd6f6459fb46a17573b821422` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:31.936923 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:31.945117 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bcd314-0600-4fe6-9167-b0d0facfc988 None admin] RBAC: Authorizing `identity:create_grant(project_id=9fd4be43faec4e4191e0e4ec4a4588c3, user_id=712c35b765b64919aa3eaaf02892590d, role_id=a93ddb1fd6f6459fb46a17573b821422)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:31.946931 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bcd314-0600-4fe6-9167-b0d0facfc988 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:32.015283 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4634f0d-08cb-429b-9a1c-dd3b3d4a0241 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:32.034923 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 406/821] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Aug 25 03:01:31 2022] PUT /identity/v3/projects/9fd4be43faec4e4191e0e4ec4a4588c3/users/712c35b765b64919aa3eaaf02892590d/roles/a93ddb1fd6f6459fb46a17573b821422 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:01:32.035716 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bcd314-0600-4fe6-9167-b0d0facfc988 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:32.095690 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6d99b9-2bca-4c36-9327-509dbc47dea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:32.096247 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6d99b9-2bca-4c36-9327-509dbc47dea5 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:32.096522 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6d99b9-2bca-4c36-9327-509dbc47dea5 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:32.096788 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6d99b9-2bca-4c36-9327-509dbc47dea5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:32.206027 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:32.215832 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6d99b9-2bca-4c36-9327-509dbc47dea5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:32.217330 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6d99b9-2bca-4c36-9327-509dbc47dea5 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:32.229625 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bcd314-0600-4fe6-9167-b0d0facfc988 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:32.262027 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:32.276041 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55b74ff-96cb-4c4d-9994-b99be15c6c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:32.276679 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55b74ff-96cb-4c4d-9994-b99be15c6c9c None admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:32.276966 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55b74ff-96cb-4c4d-9994-b99be15c6c9c None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:32.277244 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55b74ff-96cb-4c4d-9994-b99be15c6c9c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:32.278176 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55b74ff-96cb-4c4d-9994-b99be15c6c9c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:32.279391 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55b74ff-96cb-4c4d-9994-b99be15c6c9c None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:32.341007 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 416/822] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Aug 25 03:01:31 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 25 03:01:32.342085 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6d99b9-2bca-4c36-9327-509dbc47dea5 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:32.380192 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 407/823] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:01:32 2022] POST /identity/v3/endpoints => generated 346 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:01:32.383015 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55b74ff-96cb-4c4d-9994-b99be15c6c9c None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:32.513744 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6d99b9-2bca-4c36-9327-509dbc47dea5 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:32.564415 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:32.592548 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55b74ff-96cb-4c4d-9994-b99be15c6c9c None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:32.596015 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef9124b-a7e2-4c96-a61f-40eb5c332b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:32.597593 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef9124b-a7e2-4c96-a61f-40eb5c332b95 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:32.599572 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef9124b-a7e2-4c96-a61f-40eb5c332b95 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:32.600017 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef9124b-a7e2-4c96-a61f-40eb5c332b95 None admin] PATH_INFO: `/v3/services/network` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:32.601362 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef9124b-a7e2-4c96-a61f-40eb5c332b95 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:32.602812 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef9124b-a7e2-4c96-a61f-40eb5c332b95 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:32.617721 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-8ef9124b-a7e2-4c96-a61f-40eb5c332b95 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 25 03:01:32.617721 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:32.617721 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 25 03:01:32.617721 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 25 03:01:32.617721 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 25 03:01:32.617721 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 25 03:01:32.617721 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 25 03:01:32.617721 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 03:01:32.617721 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 25 03:01:32.617721 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:32.617721 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 25 03:01:32.617721 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:32.617721 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:32.617721 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:32.617721 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:32.617721 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:32.617721 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:32.618798 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:32.618798 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:32.618798 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:32.618798 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:32.618798 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:32.618798 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:32.618798 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 25 03:01:32.618798 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 25 03:01:32.618798 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 25 03:01:32.618798 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 25 03:01:32.618798 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:32.618798 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:32.618798 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:32.618798 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:32.618798 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:32.618798 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:32.619620 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:32.619620 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:32.619620 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:32.619620 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:32.619620 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:32.619620 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:32.619620 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:32.619620 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:32.619620 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:32.619620 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:32.619620 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 25 03:01:32.619620 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 03:01:32.619620 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 25 03:01:32.619620 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:32.621960 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 417/824] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Aug 25 03:01:32 2022] GET /identity/v3/services/network => generated 88 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:32.625300 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef9124b-a7e2-4c96-a61f-40eb5c332b95 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:32.645330 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11724bd2-c3f3-4120-b187-aa7462fcfe92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:32.646314 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11724bd2-c3f3-4120-b187-aa7462fcfe92 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:32.646813 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11724bd2-c3f3-4120-b187-aa7462fcfe92 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:32.647278 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11724bd2-c3f3-4120-b187-aa7462fcfe92 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:32.649601 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:32.661115 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11724bd2-c3f3-4120-b187-aa7462fcfe92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:32.664955 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11724bd2-c3f3-4120-b187-aa7462fcfe92 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:32.679766 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-11724bd2-c3f3-4120-b187-aa7462fcfe92 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:32.679766 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:32.679766 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:32.679766 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:32.679766 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:32.679766 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:32.679766 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:32.679766 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:32.679766 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:32.679766 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:32.679766 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:32.679766 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:32.679766 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 03:01:32.679766 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 03:01:32.679766 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 03:01:32.679766 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 03:01:32.680767 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:32.680767 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:32.680767 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:32.680767 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:32.680767 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:32.680767 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:32.680767 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:32.680767 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:32.680767 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:32.680767 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:32.680767 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:32.680767 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:32.680767 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:32.680767 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:32.680767 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:32.680767 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:32.680767 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 03:01:32.681642 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 03:01:32.681642 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:32.681642 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:32.684516 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 408/825] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:01:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:32.693388 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11724bd2-c3f3-4120-b187-aa7462fcfe92 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:32.760202 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef9124b-a7e2-4c96-a61f-40eb5c332b95 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:32.797448 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf2c825-facf-4e5b-8d3b-42c1a2f90690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:32.798438 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf2c825-facf-4e5b-8d3b-42c1a2f90690 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:32.798913 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf2c825-facf-4e5b-8d3b-42c1a2f90690 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:32.799516 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf2c825-facf-4e5b-8d3b-42c1a2f90690 None admin] PATH_INFO: `/v3/services` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:32.800319 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf2c825-facf-4e5b-8d3b-42c1a2f90690 None admin] RBAC: Adding query filter params (name=network) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:32.800810 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf2c825-facf-4e5b-8d3b-42c1a2f90690 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:32.802440 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf2c825-facf-4e5b-8d3b-42c1a2f90690 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:32.807594 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 418/826] 10.222.0.58 () {56 vars in 1112 bytes} [Thu Aug 25 03:01:32 2022] GET /identity/v3/services?name=network => generated 124 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:01:32.813953 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf2c825-facf-4e5b-8d3b-42c1a2f90690 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:32.835444 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11724bd2-c3f3-4120-b187-aa7462fcfe92 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:32.880982 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:32.913749 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4611fc55-13db-4fd9-b549-9b318885815d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:32.913749 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4611fc55-13db-4fd9-b549-9b318885815d None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:32.913749 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4611fc55-13db-4fd9-b549-9b318885815d None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:32.913749 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4611fc55-13db-4fd9-b549-9b318885815d None admin] PATH_INFO: `/v3/domains` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:32.914214 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4611fc55-13db-4fd9-b549-9b318885815d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:32.914214 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4611fc55-13db-4fd9-b549-9b318885815d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:32.914214 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4611fc55-13db-4fd9-b549-9b318885815d None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:32.924645 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 409/827] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Aug 25 03:01:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:32.928344 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf2c825-facf-4e5b-8d3b-42c1a2f90690 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:32.929645 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4611fc55-13db-4fd9-b549-9b318885815d None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:32.999925 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8784f70a-6e29-489d-ae85-653ab0761094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:33.001124 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8784f70a-6e29-489d-ae85-653ab0761094 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:33.001572 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8784f70a-6e29-489d-ae85-653ab0761094 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:33.002033 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8784f70a-6e29-489d-ae85-653ab0761094 None admin] PATH_INFO: `/v3/services` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:33.004156 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8784f70a-6e29-489d-ae85-653ab0761094 None admin] RBAC: Adding query filter params (type=network) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:33.008659 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8784f70a-6e29-489d-ae85-653ab0761094 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:33.008659 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8784f70a-6e29-489d-ae85-653ab0761094 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:33.016575 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:33.018207 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 419/828] 10.222.0.58 () {56 vars in 1112 bytes} [Thu Aug 25 03:01:32 2022] GET /identity/v3/services?type=network => generated 350 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:33.024964 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8784f70a-6e29-489d-ae85-653ab0761094 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:33.137150 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4611fc55-13db-4fd9-b549-9b318885815d None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:33.155890 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8784f70a-6e29-489d-ae85-653ab0761094 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:33.177066 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:33.178323 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87ad22d-1be6-4364-a8d2-6b84a61a2b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:33.178866 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ad22d-1be6-4364-a8d2-6b84a61a2b0d None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:33.179210 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ad22d-1be6-4364-a8d2-6b84a61a2b0d None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:33.179597 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ad22d-1be6-4364-a8d2-6b84a61a2b0d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:33.190669 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87ad22d-1be6-4364-a8d2-6b84a61a2b0d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:33.192126 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87ad22d-1be6-4364-a8d2-6b84a61a2b0d None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:33.198116 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005fd5a3-adcf-4fe8-8c9a-bd6dbfde644c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:33.199266 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005fd5a3-adcf-4fe8-8c9a-bd6dbfde644c None admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:33.199695 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005fd5a3-adcf-4fe8-8c9a-bd6dbfde644c None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:33.200131 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005fd5a3-adcf-4fe8-8c9a-bd6dbfde644c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:33.200933 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005fd5a3-adcf-4fe8-8c9a-bd6dbfde644c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:33.203251 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005fd5a3-adcf-4fe8-8c9a-bd6dbfde644c None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:33.205322 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-c87ad22d-1be6-4364-a8d2-6b84a61a2b0d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 03:01:33.205322 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:33.205322 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:33.205322 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:33.205322 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:33.205322 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:33.205322 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:33.205322 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:33.205322 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:33.205322 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:33.205322 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:33.205322 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:33.205322 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 25 03:01:33.205322 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 25 03:01:33.205322 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 25 03:01:33.206513 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 25 03:01:33.206513 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:33.206513 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:33.206513 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:33.206513 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:33.206513 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:33.206513 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:33.206513 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:33.206513 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:33.206513 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:33.206513 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:33.206513 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:33.206513 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:33.206513 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:33.206513 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:33.206513 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:33.206513 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:33.207646 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 25 03:01:33.207646 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 25 03:01:33.207646 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 25 03:01:33.207646 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 25 03:01:33.207646 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 25 03:01:33.207646 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 25 03:01:33.207646 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 03:01:33.207646 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:33.208787 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 410/829] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Aug 25 03:01:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 03:01:33.215701 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96289afc-9ec2-4341-b4f4-85c47958f01e None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:33.216083 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96289afc-9ec2-4341-b4f4-85c47958f01e None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:33.216437 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96289afc-9ec2-4341-b4f4-85c47958f01e None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:33.219802 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 411/830] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:01:33 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:01:33.221629 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96289afc-9ec2-4341-b4f4-85c47958f01e None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:33.285903 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 420/831] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:01:33 2022] POST /identity/v3/endpoints => generated 337 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:01:33.287403 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50acafc3-9b41-4f98-9265-3203421da9c6 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:33.287861 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50acafc3-9b41-4f98-9265-3203421da9c6 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:33.288291 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50acafc3-9b41-4f98-9265-3203421da9c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:33.322037 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-50acafc3-9b41-4f98-9265-3203421da9c6 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:01:33.326271 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-50acafc3-9b41-4f98-9265-3203421da9c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:01:33.431151 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96289afc-9ec2-4341-b4f4-85c47958f01e None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:33.438742 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:33.442414 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 421/832] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:01:33 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 03:01:33.481097 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7777ae1-9780-465c-aa95-bb2717f24347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:33.481973 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7777ae1-9780-465c-aa95-bb2717f24347 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:33.482432 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7777ae1-9780-465c-aa95-bb2717f24347 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:33.482885 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7777ae1-9780-465c-aa95-bb2717f24347 None admin] PATH_INFO: `/v3/projects` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:33.483767 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7777ae1-9780-465c-aa95-bb2717f24347 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:33.484230 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7777ae1-9780-465c-aa95-bb2717f24347 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:33.485807 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7777ae1-9780-465c-aa95-bb2717f24347 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:33.500771 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 412/833] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Aug 25 03:01:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:33.506500 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50acafc3-9b41-4f98-9265-3203421da9c6 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:33.619818 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50acafc3-9b41-4f98-9265-3203421da9c6 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:33.628466 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:33.660315 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b5bb61-532e-4b40-8836-b6f7c9e282d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:33.660931 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b5bb61-532e-4b40-8836-b6f7c9e282d9 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:33.661211 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b5bb61-532e-4b40-8836-b6f7c9e282d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:33.661481 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b5bb61-532e-4b40-8836-b6f7c9e282d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:33.662216 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b5bb61-532e-4b40-8836-b6f7c9e282d9 None admin] RBAC: Adding query filter params (role.id=62f19ea272b447ccaab1576cd9bf4fd9, scope.project.id=9fd4be43faec4e4191e0e4ec4a4588c3, user.id=0cd171bf4df94f8fab397e42835feeab) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:33.662558 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b5bb61-532e-4b40-8836-b6f7c9e282d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:33.663984 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b5bb61-532e-4b40-8836-b6f7c9e282d9 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:33.674798 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 422/834] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Aug 25 03:01:33 2022] GET /identity/v3/role_assignments?user.id=0cd171bf4df94f8fab397e42835feeab&scope.project.id=9fd4be43faec4e4191e0e4ec4a4588c3&role.id=62f19ea272b447ccaab1576cd9bf4fd9 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:33.785725 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7777ae1-9780-465c-aa95-bb2717f24347 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:33.901410 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7777ae1-9780-465c-aa95-bb2717f24347 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:33.937428 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e146391a-9d49-4353-b2ac-f77d1a39910d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:33.938344 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e146391a-9d49-4353-b2ac-f77d1a39910d None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:33.938797 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e146391a-9d49-4353-b2ac-f77d1a39910d None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:33.939217 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e146391a-9d49-4353-b2ac-f77d1a39910d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:33.949893 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e146391a-9d49-4353-b2ac-f77d1a39910d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:01:33.955549 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-e146391a-9d49-4353-b2ac-f77d1a39910d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 03:01:33.955549 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:33.955549 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:33.955549 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:33.955549 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:33.955549 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:33.955549 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:33.955549 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:33.955549 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:33.955549 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:33.955549 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:33.955549 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:33.955549 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:01:33.955549 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:01:33.955549 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:01:33.955549 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:01:33.956687 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:01:33.956687 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:01:33.956687 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:01:33.956687 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:01:33.956687 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:33.956687 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:33.956687 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:33.956687 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:33.956687 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:33.956687 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:33.956687 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:33.956687 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:33.956687 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:33.956687 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:33.956687 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:33.956687 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:33.957752 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:33.957752 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:33.957752 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:33.957752 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:33.957752 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:01:33.957752 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:01:33.957752 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:01:33.957752 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:01:33.957752 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:01:33.957752 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:01:33.957752 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 03:01:33.957752 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:33.959715 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 413/835] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Aug 25 03:01:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:33.966416 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b5bb61-532e-4b40-8836-b6f7c9e282d9 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:34.071141 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b5bb61-532e-4b40-8836-b6f7c9e282d9 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:34.107011 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558ee3c3-f272-4ee0-950b-ae5c43c551d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:34.107942 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558ee3c3-f272-4ee0-950b-ae5c43c551d8 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:34.108957 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558ee3c3-f272-4ee0-950b-ae5c43c551d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:34.109383 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558ee3c3-f272-4ee0-950b-ae5c43c551d8 None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:34.110196 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558ee3c3-f272-4ee0-950b-ae5c43c551d8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:34.110671 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558ee3c3-f272-4ee0-950b-ae5c43c551d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:34.112200 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558ee3c3-f272-4ee0-950b-ae5c43c551d8 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:34.121305 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 423/836] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Aug 25 03:01:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:34.127105 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e146391a-9d49-4353-b2ac-f77d1a39910d None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:34.141791 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dce63cf-953e-4f2f-aec3-8c1ff2783c20 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:34.142084 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dce63cf-953e-4f2f-aec3-8c1ff2783c20 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:34.142378 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dce63cf-953e-4f2f-aec3-8c1ff2783c20 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:34.143958 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 424/837] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:01:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 25 03:01:34.150085 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad78b026-c76c-4e83-ac32-9df82f0e45a1 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:34.150632 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad78b026-c76c-4e83-ac32-9df82f0e45a1 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:34.151109 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad78b026-c76c-4e83-ac32-9df82f0e45a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:34.157635 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:34.170368 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-ad78b026-c76c-4e83-ac32-9df82f0e45a1 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:01:34.172491 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-ad78b026-c76c-4e83-ac32-9df82f0e45a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:01:34.200734 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:34.261502 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 425/838] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:01:34 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 25 03:01:34.263022 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e146391a-9d49-4353-b2ac-f77d1a39910d None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:34.298602 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa164412-547a-448f-9ecd-095bda82ccae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:34.299237 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa164412-547a-448f-9ecd-095bda82ccae None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:34.299518 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa164412-547a-448f-9ecd-095bda82ccae None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:34.299791 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa164412-547a-448f-9ecd-095bda82ccae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:34.310062 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa164412-547a-448f-9ecd-095bda82ccae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:34.311620 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa164412-547a-448f-9ecd-095bda82ccae None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:34.326244 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-fa164412-547a-448f-9ecd-095bda82ccae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:34.326244 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:34.326244 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:34.326244 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:34.326244 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:34.326244 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:34.326244 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:34.326244 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:34.326244 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:34.326244 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:34.326244 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:34.326244 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:34.326244 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 03:01:34.326244 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 03:01:34.326244 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 03:01:34.326244 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 03:01:34.327240 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:34.327240 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:34.327240 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:34.327240 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:34.327240 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:34.327240 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:34.327240 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:34.327240 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:34.327240 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:34.327240 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:34.327240 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:34.327240 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:34.327240 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:34.327240 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:34.327240 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:34.327240 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:34.327240 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 03:01:34.328391 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 03:01:34.328391 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:34.328391 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:34.328391 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 414/839] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:01:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:34.335367 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad78b026-c76c-4e83-ac32-9df82f0e45a1 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:34.444442 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad78b026-c76c-4e83-ac32-9df82f0e45a1 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:34.481503 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ca9469-723a-4b89-a7b8-0f12dfe45829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:34.482159 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ca9469-723a-4b89-a7b8-0f12dfe45829 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:34.482469 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ca9469-723a-4b89-a7b8-0f12dfe45829 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:34.482742 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ca9469-723a-4b89-a7b8-0f12dfe45829 None admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:34.483413 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ca9469-723a-4b89-a7b8-0f12dfe45829 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:34.483715 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ca9469-723a-4b89-a7b8-0f12dfe45829 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:34.485081 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ca9469-723a-4b89-a7b8-0f12dfe45829 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:34.496833 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 426/840] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Aug 25 03:01:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:34.502351 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa164412-547a-448f-9ecd-095bda82ccae None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:34.549192 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ca9469-723a-4b89-a7b8-0f12dfe45829 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:34.576384 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:34.579703 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:34.605075 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa164412-547a-448f-9ecd-095bda82ccae None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:34.641183 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd4dfd7-73df-4bf6-b2ca-8d1fa8cf3b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:34.641798 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd4dfd7-73df-4bf6-b2ca-8d1fa8cf3b67 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:34.642173 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd4dfd7-73df-4bf6-b2ca-8d1fa8cf3b67 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:34.642562 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd4dfd7-73df-4bf6-b2ca-8d1fa8cf3b67 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:34.651013 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd4dfd7-73df-4bf6-b2ca-8d1fa8cf3b67 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:34.652725 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd4dfd7-73df-4bf6-b2ca-8d1fa8cf3b67 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:34.662672 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ca9469-723a-4b89-a7b8-0f12dfe45829 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:34.665817 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-4bd4dfd7-73df-4bf6-b2ca-8d1fa8cf3b67 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 25 03:01:34.665817 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:34.665817 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:34.665817 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:34.665817 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:34.665817 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:34.665817 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:34.665817 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:34.665817 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:34.665817 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:34.665817 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:34.665817 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:34.665817 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 25 03:01:34.665817 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 25 03:01:34.665817 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 25 03:01:34.665817 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 25 03:01:34.667004 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:34.667004 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:34.667004 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 25 03:01:34.667004 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 03:01:34.667004 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 25 03:01:34.667004 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 03:01:34.667004 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:34.667004 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:34.667004 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:34.667004 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:34.667004 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:34.667004 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:34.667004 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:34.667004 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:34.667004 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:34.667004 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:34.667004 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:34.667004 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:34.668285 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:34.668285 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:34.668285 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 25 03:01:34.668285 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 25 03:01:34.668285 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 25 03:01:34.668285 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 25 03:01:34.668285 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 25 03:01:34.668285 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 25 03:01:34.668285 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 25 03:01:34.668285 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:34.669528 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 415/841] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Aug 25 03:01:34 2022] GET /identity/v3/users/swift => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:34.675460 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd4dfd7-73df-4bf6-b2ca-8d1fa8cf3b67 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:34.706236 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26690823-b052-432d-8ca9-5d644feb4376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:34.707206 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26690823-b052-432d-8ca9-5d644feb4376 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:34.707680 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26690823-b052-432d-8ca9-5d644feb4376 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:34.708096 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26690823-b052-432d-8ca9-5d644feb4376 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:34.720741 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26690823-b052-432d-8ca9-5d644feb4376 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:01:34.726063 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-26690823-b052-432d-8ca9-5d644feb4376 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 25 03:01:34.726063 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:34.726063 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:34.726063 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:34.726063 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:34.726063 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:34.726063 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:34.726063 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:34.726063 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:34.726063 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:34.726063 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:34.726063 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:34.726063 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:01:34.726063 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:01:34.726063 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:01:34.726063 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:01:34.727024 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:01:34.727024 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:01:34.727024 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:01:34.727024 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:01:34.727024 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:34.727024 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:34.727024 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:34.727024 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:34.727024 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:34.727024 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:34.727024 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:34.727024 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:34.727024 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:34.727024 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:34.727024 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:34.727024 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:34.727857 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:34.727857 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:34.727857 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:34.727857 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:34.727857 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:01:34.727857 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:01:34.727857 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:01:34.727857 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:01:34.727857 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:01:34.727857 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:01:34.727857 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 25 03:01:34.727857 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:34.729264 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 427/842] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Aug 25 03:01:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 03:01:34.730867 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf6c3df-6132-4dfe-960b-94babc4bf803 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:34.731300 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf6c3df-6132-4dfe-960b-94babc4bf803 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:34.731866 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf6c3df-6132-4dfe-960b-94babc4bf803 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:34.733524 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 428/843] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:01:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:01:34.734477 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf6c3df-6132-4dfe-960b-94babc4bf803 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:34.798607 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd4dfd7-73df-4bf6-b2ca-8d1fa8cf3b67 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:34.825168 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:34.834607 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c34b65a-485c-423d-826f-63d1c8254841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:34.835151 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c34b65a-485c-423d-826f-63d1c8254841 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:34.835428 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c34b65a-485c-423d-826f-63d1c8254841 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:34.835697 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c34b65a-485c-423d-826f-63d1c8254841 None admin] PATH_INFO: `/v3/users` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:34.836421 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c34b65a-485c-423d-826f-63d1c8254841 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:34.836728 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c34b65a-485c-423d-826f-63d1c8254841 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:34.838155 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c34b65a-485c-423d-826f-63d1c8254841 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:34.839798 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:34.847335 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf6c3df-6132-4dfe-960b-94babc4bf803 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:34.859527 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 416/844] 10.222.0.58 () {56 vars in 1135 bytes} [Thu Aug 25 03:01:34 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:34.861057 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d408ef8-4945-40cd-82ef-f563dbc06627 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:34.861346 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d408ef8-4945-40cd-82ef-f563dbc06627 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:34.861619 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d408ef8-4945-40cd-82ef-f563dbc06627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:34.879556 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-5d408ef8-4945-40cd-82ef-f563dbc06627 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:01:34.881189 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-5d408ef8-4945-40cd-82ef-f563dbc06627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:01:34.885096 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb858a78-2332-4063-85b6-3435bce7e1d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:34.885717 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb858a78-2332-4063-85b6-3435bce7e1d2 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:34.886001 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb858a78-2332-4063-85b6-3435bce7e1d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:34.886359 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb858a78-2332-4063-85b6-3435bce7e1d2 None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:34.887030 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb858a78-2332-4063-85b6-3435bce7e1d2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:34.887336 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb858a78-2332-4063-85b6-3435bce7e1d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:34.889119 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb858a78-2332-4063-85b6-3435bce7e1d2 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:34.905383 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 429/845] 10.222.0.58 () {56 vars in 1103 bytes} [Thu Aug 25 03:01:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:34.905383 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb858a78-2332-4063-85b6-3435bce7e1d2 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:34.950728 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 417/846] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:01:34 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 03:01:34.951880 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d408ef8-4945-40cd-82ef-f563dbc06627 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:35.052349 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb858a78-2332-4063-85b6-3435bce7e1d2 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:35.092520 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d408ef8-4945-40cd-82ef-f563dbc06627 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:35.098449 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dbc803-7d21-49c7-81bb-88435f85a996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:35.099833 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dbc803-7d21-49c7-81bb-88435f85a996 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:35.101008 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dbc803-7d21-49c7-81bb-88435f85a996 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:35.101499 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dbc803-7d21-49c7-81bb-88435f85a996 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:35.117617 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:35.122981 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:35.131020 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748bb369-25cf-4db9-8584-0ded8e054c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:35.131723 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748bb369-25cf-4db9-8584-0ded8e054c70 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:35.132006 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748bb369-25cf-4db9-8584-0ded8e054c70 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:35.134985 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748bb369-25cf-4db9-8584-0ded8e054c70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:35.142633 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748bb369-25cf-4db9-8584-0ded8e054c70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:35.144269 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748bb369-25cf-4db9-8584-0ded8e054c70 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:35.159230 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-748bb369-25cf-4db9-8584-0ded8e054c70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:35.159230 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:35.159230 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:35.159230 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:35.159230 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:35.159230 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:35.159230 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:35.159230 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:35.159230 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:35.159230 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:35.159230 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:35.159230 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:35.159230 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 03:01:35.159230 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 03:01:35.159230 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 03:01:35.159230 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 03:01:35.160282 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:35.160282 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:35.160282 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:35.160282 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:35.160282 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:35.160282 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:35.160282 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:35.160282 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:35.160282 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:35.160282 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:35.160282 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:35.160282 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:35.160282 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:35.160282 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:35.160282 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:35.160282 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:35.160282 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 03:01:35.161158 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 03:01:35.161158 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:35.161158 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:35.164882 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 418/847] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:01:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:35.167183 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748bb369-25cf-4db9-8584-0ded8e054c70 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:35.189466 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dbc803-7d21-49c7-81bb-88435f85a996 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:35.191478 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dbc803-7d21-49c7-81bb-88435f85a996 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:35.275625 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748bb369-25cf-4db9-8584-0ded8e054c70 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:35.310680 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b69d01-536b-407b-ac52-a5f072623ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:35.311290 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b69d01-536b-407b-ac52-a5f072623ca9 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:35.312127 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b69d01-536b-407b-ac52-a5f072623ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:35.312403 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b69d01-536b-407b-ac52-a5f072623ca9 None admin] PATH_INFO: `/v3/domains` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:35.313045 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b69d01-536b-407b-ac52-a5f072623ca9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:35.313348 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b69d01-536b-407b-ac52-a5f072623ca9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:35.318803 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 430/848] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Aug 25 03:01:34 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 25 03:01:35.318936 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b69d01-536b-407b-ac52-a5f072623ca9 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:35.319014 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dbc803-7d21-49c7-81bb-88435f85a996 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:35.326573 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 419/849] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Aug 25 03:01:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:01:35.331537 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b69d01-536b-407b-ac52-a5f072623ca9 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:35.368770 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:35.371655 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:35.443142 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dbc803-7d21-49c7-81bb-88435f85a996 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:35.447703 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b69d01-536b-407b-ac52-a5f072623ca9 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:35.481866 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a639ae5-d33e-462a-b6d2-bf1b9dd4b6f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:35.482532 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a639ae5-d33e-462a-b6d2-bf1b9dd4b6f5 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:35.482814 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a639ae5-d33e-462a-b6d2-bf1b9dd4b6f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:35.483091 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a639ae5-d33e-462a-b6d2-bf1b9dd4b6f5 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:35.484327 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a639ae5-d33e-462a-b6d2-bf1b9dd4b6f5 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:35.485283 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f891621-a408-445c-aa4b-03c68afdc9d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:35.485700 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a639ae5-d33e-462a-b6d2-bf1b9dd4b6f5 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:35.485918 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f891621-a408-445c-aa4b-03c68afdc9d3 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:35.486232 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f891621-a408-445c-aa4b-03c68afdc9d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:35.486538 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f891621-a408-445c-aa4b-03c68afdc9d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:35.494738 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-8a639ae5-d33e-462a-b6d2-bf1b9dd4b6f5 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 25 03:01:35.494738 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:35.494738 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 25 03:01:35.494738 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 25 03:01:35.494738 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 25 03:01:35.494738 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 25 03:01:35.494738 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 25 03:01:35.494738 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 03:01:35.494738 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 25 03:01:35.494738 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:35.494738 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 25 03:01:35.494738 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:35.494738 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:35.494738 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:35.494738 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:35.494738 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:35.494738 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:35.495698 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:35.495698 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:35.495698 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:35.495698 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:35.495698 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:35.495698 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:35.495698 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 25 03:01:35.495698 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 25 03:01:35.495698 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 25 03:01:35.495698 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 25 03:01:35.495698 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:35.495698 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:35.495698 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:35.495698 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:35.495698 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:35.495698 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:35.496524 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:35.496524 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:35.496524 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:35.496524 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:35.496524 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:35.496524 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:35.496524 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:35.496524 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:35.496524 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:35.496524 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:35.496524 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 25 03:01:35.496524 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 03:01:35.496524 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 25 03:01:35.496524 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:35.500451 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f891621-a408-445c-aa4b-03c68afdc9d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:35.501244 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 431/850] 10.222.0.58 () {56 vars in 1114 bytes} [Thu Aug 25 03:01:35 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:35.501904 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f891621-a408-445c-aa4b-03c68afdc9d3 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:35.502061 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a639ae5-d33e-462a-b6d2-bf1b9dd4b6f5 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:35.519418 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-5f891621-a408-445c-aa4b-03c68afdc9d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:35.519418 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:35.519418 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:35.519418 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:35.519418 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:35.519418 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:35.519418 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:35.519418 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:35.519418 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:35.519418 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:35.519418 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:35.519418 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:35.519418 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 03:01:35.519418 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 03:01:35.519418 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 03:01:35.519418 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 03:01:35.520486 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:35.520486 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:35.520486 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:35.520486 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:35.520486 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:35.520486 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:35.520486 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:35.520486 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:35.520486 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:35.520486 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:35.520486 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:35.520486 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:35.520486 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:35.520486 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:35.520486 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:35.520486 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:35.520486 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 03:01:35.521369 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 03:01:35.521369 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:35.521369 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:35.521369 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 420/851] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:01:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:35.521369 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f891621-a408-445c-aa4b-03c68afdc9d3 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:35.616478 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a639ae5-d33e-462a-b6d2-bf1b9dd4b6f5 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:35.616679 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:35.627323 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:35.633142 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f891621-a408-445c-aa4b-03c68afdc9d3 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:35.656883 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77eb124-9f6b-499f-9c1a-20196c902124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:35.657812 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77eb124-9f6b-499f-9c1a-20196c902124 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:35.658239 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77eb124-9f6b-499f-9c1a-20196c902124 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:35.658691 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77eb124-9f6b-499f-9c1a-20196c902124 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:35.666341 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77eb124-9f6b-499f-9c1a-20196c902124 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:35.668116 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6094d7-a287-40a6-ba0d-f5f196f18be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:35.668289 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77eb124-9f6b-499f-9c1a-20196c902124 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:35.668759 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6094d7-a287-40a6-ba0d-f5f196f18be8 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:35.669043 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6094d7-a287-40a6-ba0d-f5f196f18be8 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:35.669338 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6094d7-a287-40a6-ba0d-f5f196f18be8 None admin] PATH_INFO: `/v3/services` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:35.669981 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6094d7-a287-40a6-ba0d-f5f196f18be8 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:35.670307 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6094d7-a287-40a6-ba0d-f5f196f18be8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:35.672890 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6094d7-a287-40a6-ba0d-f5f196f18be8 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:35.677411 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 421/852] 10.222.0.58 () {56 vars in 1114 bytes} [Thu Aug 25 03:01:35 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:35.682766 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6094d7-a287-40a6-ba0d-f5f196f18be8 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:35.684297 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-f77eb124-9f6b-499f-9c1a-20196c902124 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 25 03:01:35.684297 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:35.684297 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:35.684297 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:35.684297 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:35.684297 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:35.684297 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:35.684297 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:35.684297 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:35.684297 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:35.684297 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:35.684297 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:35.684297 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 25 03:01:35.684297 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 25 03:01:35.684297 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 25 03:01:35.684297 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 25 03:01:35.685202 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:35.685202 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:35.685202 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 25 03:01:35.685202 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 03:01:35.685202 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 25 03:01:35.685202 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 03:01:35.685202 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:35.685202 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:35.685202 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:35.685202 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:35.685202 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:35.685202 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:35.685202 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:35.685202 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:35.685202 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:35.685202 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:35.685202 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:35.685202 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:35.686110 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:35.686110 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:35.686110 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 25 03:01:35.686110 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 25 03:01:35.686110 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 25 03:01:35.686110 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 25 03:01:35.686110 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 25 03:01:35.686110 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 25 03:01:35.686110 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 25 03:01:35.686110 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:35.686110 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 432/853] 10.222.0.58 () {56 vars in 1117 bytes} [Thu Aug 25 03:01:35 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:35.686110 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77eb124-9f6b-499f-9c1a-20196c902124 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:35.794535 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77eb124-9f6b-499f-9c1a-20196c902124 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:35.798127 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6094d7-a287-40a6-ba0d-f5f196f18be8 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:35.827608 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fe76c3-6ab8-4897-af8f-8e018e1802f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:35.828469 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fe76c3-6ab8-4897-af8f-8e018e1802f6 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:35.828751 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fe76c3-6ab8-4897-af8f-8e018e1802f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:35.829020 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fe76c3-6ab8-4897-af8f-8e018e1802f6 None admin] PATH_INFO: `/v3/services` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:35.829643 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fe76c3-6ab8-4897-af8f-8e018e1802f6 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:35.829950 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fe76c3-6ab8-4897-af8f-8e018e1802f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:35.831070 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b3fe6b-3eaa-4945-877d-143ae838633a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:35.831547 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fe76c3-6ab8-4897-af8f-8e018e1802f6 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:35.831673 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b3fe6b-3eaa-4945-877d-143ae838633a None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:35.831954 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b3fe6b-3eaa-4945-877d-143ae838633a None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:35.832228 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b3fe6b-3eaa-4945-877d-143ae838633a None admin] PATH_INFO: `/v3/domains` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:35.832869 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b3fe6b-3eaa-4945-877d-143ae838633a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:35.833420 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b3fe6b-3eaa-4945-877d-143ae838633a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:35.834556 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b3fe6b-3eaa-4945-877d-143ae838633a None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:35.844168 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 433/854] 10.222.0.58 () {56 vars in 1114 bytes} [Thu Aug 25 03:01:35 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:35.844168 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fe76c3-6ab8-4897-af8f-8e018e1802f6 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:35.847497 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 422/855] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Aug 25 03:01:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:35.850076 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a73ac9-6467-4f23-bb68-bad07e713b55 None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:35.850679 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a73ac9-6467-4f23-bb68-bad07e713b55 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:35.851215 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a73ac9-6467-4f23-bb68-bad07e713b55 None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:35.853524 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 423/856] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:01:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:01:35.855112 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a73ac9-6467-4f23-bb68-bad07e713b55 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:35.870717 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:35.884114 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:35.958358 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fe76c3-6ab8-4897-af8f-8e018e1802f6 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:35.973137 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a73ac9-6467-4f23-bb68-bad07e713b55 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:36.004646 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53905a9-4e81-4e48-b558-9a630e84ab0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:36.005617 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53905a9-4e81-4e48-b558-9a630e84ab0a None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:36.005968 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53905a9-4e81-4e48-b558-9a630e84ab0a None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:36.006555 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53905a9-4e81-4e48-b558-9a630e84ab0a None admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:36.007645 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53905a9-4e81-4e48-b558-9a630e84ab0a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:36.008103 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53905a9-4e81-4e48-b558-9a630e84ab0a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:36.009537 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53905a9-4e81-4e48-b558-9a630e84ab0a None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:36.011871 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3937dc5c-4120-43a2-8022-63eeab311a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:36.012503 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3937dc5c-4120-43a2-8022-63eeab311a6a None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:36.012783 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3937dc5c-4120-43a2-8022-63eeab311a6a None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:36.013054 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3937dc5c-4120-43a2-8022-63eeab311a6a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:36.021001 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3937dc5c-4120-43a2-8022-63eeab311a6a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:36.022479 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3937dc5c-4120-43a2-8022-63eeab311a6a None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:36.032826 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 434/857] 10.222.0.58 () {56 vars in 1149 bytes} [Thu Aug 25 03:01:35 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:36.032826 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e356111e-2395-4c01-924d-661f8dc53f9e None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:36.032826 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e356111e-2395-4c01-924d-661f8dc53f9e None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:36.032826 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e356111e-2395-4c01-924d-661f8dc53f9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:36.039744 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-3937dc5c-4120-43a2-8022-63eeab311a6a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 03:01:36.039744 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:36.039744 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:36.039744 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:36.039744 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:36.039744 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:36.039744 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:36.039744 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:36.039744 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:36.039744 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:36.039744 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:36.039744 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:36.039744 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 25 03:01:36.039744 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 25 03:01:36.039744 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 25 03:01:36.040651 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 25 03:01:36.040651 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:36.040651 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:36.040651 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:36.040651 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:36.040651 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:36.040651 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:36.040651 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:36.040651 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:36.040651 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:36.040651 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:36.040651 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:36.040651 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:36.040651 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:36.040651 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:36.040651 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:36.040651 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:36.044615 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 25 03:01:36.044615 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 25 03:01:36.044615 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 25 03:01:36.044615 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 25 03:01:36.044615 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 25 03:01:36.044615 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 25 03:01:36.044615 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 03:01:36.044615 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:36.044615 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 424/858] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Aug 25 03:01:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:36.044615 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3937dc5c-4120-43a2-8022-63eeab311a6a None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:36.050821 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-e356111e-2395-4c01-924d-661f8dc53f9e None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:01:36.052021 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-e356111e-2395-4c01-924d-661f8dc53f9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:01:36.121450 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:36.127080 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 435/859] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:01:36 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 03:01:36.127933 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e356111e-2395-4c01-924d-661f8dc53f9e None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:36.150546 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:36.171242 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3937dc5c-4120-43a2-8022-63eeab311a6a None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:36.216420 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a9c2e2-ad8b-4d5a-a53b-b590ae32006e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:36.217973 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a9c2e2-ad8b-4d5a-a53b-b590ae32006e None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:36.219228 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a9c2e2-ad8b-4d5a-a53b-b590ae32006e None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:36.219691 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a9c2e2-ad8b-4d5a-a53b-b590ae32006e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:36.236952 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e356111e-2395-4c01-924d-661f8dc53f9e None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:36.275725 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b4aaf9-c1b7-47ce-af18-601fd3514841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:36.276440 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b4aaf9-c1b7-47ce-af18-601fd3514841 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:36.277080 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b4aaf9-c1b7-47ce-af18-601fd3514841 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:36.277483 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b4aaf9-c1b7-47ce-af18-601fd3514841 None admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:36.278307 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b4aaf9-c1b7-47ce-af18-601fd3514841 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:36.278621 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b4aaf9-c1b7-47ce-af18-601fd3514841 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:36.280125 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b4aaf9-c1b7-47ce-af18-601fd3514841 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:36.291915 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 436/860] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Aug 25 03:01:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:36.298020 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b4aaf9-c1b7-47ce-af18-601fd3514841 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:36.312175 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a9c2e2-ad8b-4d5a-a53b-b590ae32006e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:36.313823 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a9c2e2-ad8b-4d5a-a53b-b590ae32006e None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:36.399624 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:36.400522 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:36.404144 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b4aaf9-c1b7-47ce-af18-601fd3514841 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:36.430975 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 425/861] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Aug 25 03:01:36 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 25 03:01:36.432239 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a9c2e2-ad8b-4d5a-a53b-b590ae32006e None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:36.444924 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5068931-cf9b-46f2-9866-cb1726404f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:36.445897 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5068931-cf9b-46f2-9866-cb1726404f56 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:36.446184 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5068931-cf9b-46f2-9866-cb1726404f56 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:36.446494 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5068931-cf9b-46f2-9866-cb1726404f56 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:36.447233 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5068931-cf9b-46f2-9866-cb1726404f56 None admin] RBAC: Adding query filter params (role.id=62f19ea272b447ccaab1576cd9bf4fd9, scope.project.id=9fd4be43faec4e4191e0e4ec4a4588c3, user.id=5047e1a7cd2d416d90b2d8c5a6596522) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:36.447545 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5068931-cf9b-46f2-9866-cb1726404f56 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:36.448994 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5068931-cf9b-46f2-9866-cb1726404f56 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:36.456066 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 437/862] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Aug 25 03:01:36 2022] GET /identity/v3/role_assignments?user.id=5047e1a7cd2d416d90b2d8c5a6596522&scope.project.id=9fd4be43faec4e4191e0e4ec4a4588c3&role.id=62f19ea272b447ccaab1576cd9bf4fd9 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:36.457173 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5068931-cf9b-46f2-9866-cb1726404f56 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:36.544125 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a9c2e2-ad8b-4d5a-a53b-b590ae32006e None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:36.595773 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bc5fa0-55c2-4a12-bf2b-1df0abf3b235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:36.596418 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bc5fa0-55c2-4a12-bf2b-1df0abf3b235 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:36.596701 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bc5fa0-55c2-4a12-bf2b-1df0abf3b235 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:36.596976 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bc5fa0-55c2-4a12-bf2b-1df0abf3b235 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:36.597570 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bc5fa0-55c2-4a12-bf2b-1df0abf3b235 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:36.599000 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bc5fa0-55c2-4a12-bf2b-1df0abf3b235 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:36.608995 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-d8bc5fa0-55c2-4a12-bf2b-1df0abf3b235 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 25 03:01:36.608995 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:36.608995 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 25 03:01:36.608995 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 25 03:01:36.608995 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 25 03:01:36.608995 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 25 03:01:36.608995 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 25 03:01:36.608995 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 03:01:36.608995 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 25 03:01:36.608995 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:36.608995 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 25 03:01:36.608995 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:36.608995 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:36.608995 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:36.608995 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:36.608995 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:36.609985 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:36.609985 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:36.609985 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:36.609985 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:36.609985 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:36.609985 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:36.609985 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:36.609985 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 25 03:01:36.609985 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 25 03:01:36.609985 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 25 03:01:36.609985 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 25 03:01:36.609985 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:36.609985 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:36.609985 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:36.609985 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:36.609985 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:36.609985 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:36.610942 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:36.610942 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:36.610942 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:36.610942 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:36.610942 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:36.610942 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:36.610942 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:36.610942 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:36.610942 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:36.610942 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:36.610942 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 25 03:01:36.610942 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 03:01:36.610942 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 25 03:01:36.610942 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:36.614803 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5068931-cf9b-46f2-9866-cb1726404f56 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:36.615108 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 426/863] 10.222.0.58 () {56 vars in 1132 bytes} [Thu Aug 25 03:01:36 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:36.616853 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bc5fa0-55c2-4a12-bf2b-1df0abf3b235 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:36.654040 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:36.658066 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6168fa14-1924-465e-ad80-6b7e1cb34272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:36.658692 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6168fa14-1924-465e-ad80-6b7e1cb34272 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:36.659083 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6168fa14-1924-465e-ad80-6b7e1cb34272 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:36.659309 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6168fa14-1924-465e-ad80-6b7e1cb34272 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:36.672366 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6168fa14-1924-465e-ad80-6b7e1cb34272 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:36.674034 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6168fa14-1924-465e-ad80-6b7e1cb34272 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:36.677211 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:36.691506 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-6168fa14-1924-465e-ad80-6b7e1cb34272 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:36.691506 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:36.691506 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:36.691506 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:36.691506 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:36.691506 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:36.691506 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:36.691506 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:36.691506 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:36.691506 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:36.691506 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:36.691506 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:36.691506 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 03:01:36.691506 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 03:01:36.691506 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 03:01:36.691506 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 03:01:36.692508 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:36.692508 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:36.692508 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:36.692508 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:36.692508 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:36.692508 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:36.692508 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:36.692508 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:36.692508 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:36.692508 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:36.692508 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:36.692508 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:36.692508 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:36.692508 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:36.692508 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:36.692508 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:36.692508 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 03:01:36.693869 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 03:01:36.693869 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:36.693869 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:36.693869 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 438/864] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:01:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:36.698273 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6168fa14-1924-465e-ad80-6b7e1cb34272 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:36.747720 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bc5fa0-55c2-4a12-bf2b-1df0abf3b235 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:36.794255 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca4fe7a-2357-4c66-9bfd-e8bb848f68c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:36.795028 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca4fe7a-2357-4c66-9bfd-e8bb848f68c8 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:36.795378 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca4fe7a-2357-4c66-9bfd-e8bb848f68c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:36.795720 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca4fe7a-2357-4c66-9bfd-e8bb848f68c8 None admin] PATH_INFO: `/v3/services` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:36.797512 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca4fe7a-2357-4c66-9bfd-e8bb848f68c8 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:36.797891 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca4fe7a-2357-4c66-9bfd-e8bb848f68c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:36.799633 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca4fe7a-2357-4c66-9bfd-e8bb848f68c8 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:36.807548 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 427/865] 10.222.0.58 () {56 vars in 1126 bytes} [Thu Aug 25 03:01:36 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:36.810264 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6168fa14-1924-465e-ad80-6b7e1cb34272 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:36.813140 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca4fe7a-2357-4c66-9bfd-e8bb848f68c8 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:36.857505 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7cd54e-8b9a-410e-a597-222b0f046348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:36.861013 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7cd54e-8b9a-410e-a597-222b0f046348 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:36.861296 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7cd54e-8b9a-410e-a597-222b0f046348 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:36.861575 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7cd54e-8b9a-410e-a597-222b0f046348 None admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:36.862242 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7cd54e-8b9a-410e-a597-222b0f046348 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:36.862576 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7cd54e-8b9a-410e-a597-222b0f046348 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:36.863961 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7cd54e-8b9a-410e-a597-222b0f046348 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:36.883613 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 439/866] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Aug 25 03:01:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:01:36.888709 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7cd54e-8b9a-410e-a597-222b0f046348 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:36.926187 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:36.930199 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca4fe7a-2357-4c66-9bfd-e8bb848f68c8 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:36.971509 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:36.976113 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78fb11a-f0b0-4e1c-b83c-412d83475aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:36.977229 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78fb11a-f0b0-4e1c-b83c-412d83475aca None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:36.977888 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78fb11a-f0b0-4e1c-b83c-412d83475aca None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:36.978719 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78fb11a-f0b0-4e1c-b83c-412d83475aca None admin] PATH_INFO: `/v3/services` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:36.980016 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78fb11a-f0b0-4e1c-b83c-412d83475aca None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:36.980536 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78fb11a-f0b0-4e1c-b83c-412d83475aca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:36.982338 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78fb11a-f0b0-4e1c-b83c-412d83475aca None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:36.990098 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 428/867] 10.222.0.58 () {56 vars in 1126 bytes} [Thu Aug 25 03:01:36 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:01:37.023441 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7cd54e-8b9a-410e-a597-222b0f046348 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:37.065676 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac6e127-549c-450b-98d5-0cf11153c60c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:37.066726 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac6e127-549c-450b-98d5-0cf11153c60c None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:37.067234 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac6e127-549c-450b-98d5-0cf11153c60c None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:37.067725 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac6e127-549c-450b-98d5-0cf11153c60c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:37.075913 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac6e127-549c-450b-98d5-0cf11153c60c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:37.077753 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac6e127-549c-450b-98d5-0cf11153c60c None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:37.090818 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-6ac6e127-549c-450b-98d5-0cf11153c60c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 03:01:37.090818 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:37.090818 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:37.090818 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:37.090818 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:37.090818 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:37.090818 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:37.090818 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:37.090818 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:37.090818 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:37.090818 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:37.090818 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:37.090818 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 25 03:01:37.090818 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 25 03:01:37.090818 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 25 03:01:37.091776 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 25 03:01:37.091776 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:37.091776 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:37.091776 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:37.091776 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:37.091776 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:37.091776 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:37.091776 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:37.091776 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:37.091776 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:37.091776 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:37.091776 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:37.091776 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:37.091776 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:37.091776 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:37.091776 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:37.091776 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:37.093245 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 25 03:01:37.093245 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 25 03:01:37.093245 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 25 03:01:37.093245 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 25 03:01:37.093245 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 25 03:01:37.093245 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 25 03:01:37.093245 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 03:01:37.093245 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:37.095866 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 440/868] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Aug 25 03:01:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 03:01:37.103943 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78fb11a-f0b0-4e1c-b83c-412d83475aca None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:37.212010 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78fb11a-f0b0-4e1c-b83c-412d83475aca None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:37.254305 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8991aa-b9a7-482b-9bde-384df5bc4d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:37.254305 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8991aa-b9a7-482b-9bde-384df5bc4d7f None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:37.254305 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8991aa-b9a7-482b-9bde-384df5bc4d7f None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:37.254305 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8991aa-b9a7-482b-9bde-384df5bc4d7f None admin] PATH_INFO: `/v3/projects` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:37.254729 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8991aa-b9a7-482b-9bde-384df5bc4d7f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:37.254729 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8991aa-b9a7-482b-9bde-384df5bc4d7f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:37.254729 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8991aa-b9a7-482b-9bde-384df5bc4d7f None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:37.271902 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 429/869] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Aug 25 03:01:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:37.274411 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac6e127-549c-450b-98d5-0cf11153c60c None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:37.380721 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac6e127-549c-450b-98d5-0cf11153c60c None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:37.397897 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:37.420366 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8281ba-3258-4c64-8045-6a2e50570ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:37.421008 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8281ba-3258-4c64-8045-6a2e50570ac2 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:37.421292 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8281ba-3258-4c64-8045-6a2e50570ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:37.421569 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8281ba-3258-4c64-8045-6a2e50570ac2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:37.422321 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8281ba-3258-4c64-8045-6a2e50570ac2 None admin] RBAC: Adding query filter params (role.id=a93ddb1fd6f6459fb46a17573b821422, scope.project.id=9fd4be43faec4e4191e0e4ec4a4588c3, user.id=712c35b765b64919aa3eaaf02892590d) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:37.423586 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8281ba-3258-4c64-8045-6a2e50570ac2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:37.424118 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8281ba-3258-4c64-8045-6a2e50570ac2 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:37.430585 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 441/870] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Aug 25 03:01:37 2022] GET /identity/v3/role_assignments?user.id=712c35b765b64919aa3eaaf02892590d&scope.project.id=9fd4be43faec4e4191e0e4ec4a4588c3&role.id=a93ddb1fd6f6459fb46a17573b821422 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:37.768574 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee347d3-f857-4303-8446-f7b0ef7fadee None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:37.768881 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee347d3-f857-4303-8446-f7b0ef7fadee None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:37.769131 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee347d3-f857-4303-8446-f7b0ef7fadee None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:37.770493 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 430/871] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:01:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:01:37.783492 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b42a34c-bb8e-4420-80b1-e4220efe934b None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:37.783876 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b42a34c-bb8e-4420-80b1-e4220efe934b None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:37.784225 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b42a34c-bb8e-4420-80b1-e4220efe934b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:37.815246 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-7b42a34c-bb8e-4420-80b1-e4220efe934b None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:01:37.816447 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-7b42a34c-bb8e-4420-80b1-e4220efe934b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:01:37.908310 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 442/872] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:01:37 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 03:01:38.265823 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee347d3-f857-4303-8446-f7b0ef7fadee None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:38.294017 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:38.395481 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee347d3-f857-4303-8446-f7b0ef7fadee None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:38.433744 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554f2730-762a-4c4f-8251-b0b6449d61df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:38.434687 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554f2730-762a-4c4f-8251-b0b6449d61df None admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:38.435134 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554f2730-762a-4c4f-8251-b0b6449d61df None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:38.435638 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554f2730-762a-4c4f-8251-b0b6449d61df None admin] PATH_INFO: `/v3/services` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:38.436462 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554f2730-762a-4c4f-8251-b0b6449d61df None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:38.437971 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554f2730-762a-4c4f-8251-b0b6449d61df None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:38.503062 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 431/873] 10.222.0.58 () {60 vars in 1141 bytes} [Thu Aug 25 03:01:38 2022] POST /identity/v3/services => generated 251 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:01:38.615661 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d32afe9-8203-4644-9ed5-81911fd5ed3c None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:38.616159 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d32afe9-8203-4644-9ed5-81911fd5ed3c None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:38.616159 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d32afe9-8203-4644-9ed5-81911fd5ed3c None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:38.617786 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 443/874] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:01:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:01:38.623683 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df7e33c-a83e-47af-813c-4dad6ae4ab06 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:38.624556 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df7e33c-a83e-47af-813c-4dad6ae4ab06 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:38.624838 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df7e33c-a83e-47af-813c-4dad6ae4ab06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:38.648654 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-2df7e33c-a83e-47af-813c-4dad6ae4ab06 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:01:38.650163 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-2df7e33c-a83e-47af-813c-4dad6ae4ab06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:01:38.728909 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 432/875] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:01:38 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 03:01:38.999810 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d32afe9-8203-4644-9ed5-81911fd5ed3c None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:39.105355 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d32afe9-8203-4644-9ed5-81911fd5ed3c None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:39.117048 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:39.141190 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eaff76-9c7c-4169-a421-99351b5e4618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:39.141837 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eaff76-9c7c-4169-a421-99351b5e4618 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:39.142120 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eaff76-9c7c-4169-a421-99351b5e4618 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:39.142432 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eaff76-9c7c-4169-a421-99351b5e4618 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:39.143029 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9eaff76-9c7c-4169-a421-99351b5e4618 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:39.144594 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9eaff76-9c7c-4169-a421-99351b5e4618 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:39.153760 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-e9eaff76-9c7c-4169-a421-99351b5e4618 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 25 03:01:39.153760 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:39.153760 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 25 03:01:39.153760 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 25 03:01:39.153760 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 25 03:01:39.153760 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 25 03:01:39.153760 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 25 03:01:39.153760 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 03:01:39.153760 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 25 03:01:39.153760 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:39.153760 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 25 03:01:39.153760 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:39.153760 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:39.153760 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:39.153760 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:39.153760 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:39.153760 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:39.154852 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:39.154852 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:39.154852 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:39.154852 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:39.154852 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:39.154852 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:39.154852 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 25 03:01:39.154852 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 25 03:01:39.154852 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 25 03:01:39.154852 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 25 03:01:39.154852 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:39.154852 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:39.154852 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:39.154852 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:39.154852 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:39.154852 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:39.155734 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:39.155734 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:39.155734 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:39.155734 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:39.155734 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:39.155734 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:39.155734 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:39.155734 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:39.155734 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:39.155734 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:39.155734 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 25 03:01:39.155734 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 03:01:39.155734 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 25 03:01:39.155734 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:39.160084 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 444/876] 10.222.0.58 () {56 vars in 1126 bytes} [Thu Aug 25 03:01:38 2022] GET /identity/v3/services/object-store => generated 93 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:39.161636 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df7e33c-a83e-47af-813c-4dad6ae4ab06 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:39.182069 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f3b764-0427-4fc0-857e-713a47509052 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:39.182759 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f3b764-0427-4fc0-857e-713a47509052 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:39.183269 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f3b764-0427-4fc0-857e-713a47509052 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:39.185979 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 445/877] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:01:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 25 03:01:39.188083 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:39.195198 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ed8b3-8f89-4585-b60d-3b52ada22e4a None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:39.196102 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ed8b3-8f89-4585-b60d-3b52ada22e4a None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:39.196898 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ed8b3-8f89-4585-b60d-3b52ada22e4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:39.222797 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-fe9ed8b3-8f89-4585-b60d-3b52ada22e4a None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:01:39.225330 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-fe9ed8b3-8f89-4585-b60d-3b52ada22e4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:01:39.278127 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df7e33c-a83e-47af-813c-4dad6ae4ab06 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:39.318425 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 446/878] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:01:39 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 25 03:01:39.330561 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a57d19-6695-460e-96de-f7ebadcc92cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:39.331436 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a57d19-6695-460e-96de-f7ebadcc92cc None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:39.331921 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a57d19-6695-460e-96de-f7ebadcc92cc None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:39.332435 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a57d19-6695-460e-96de-f7ebadcc92cc None admin] PATH_INFO: `/v3/services` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:39.333343 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a57d19-6695-460e-96de-f7ebadcc92cc None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:39.333862 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a57d19-6695-460e-96de-f7ebadcc92cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:39.335535 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a57d19-6695-460e-96de-f7ebadcc92cc None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:39.345134 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 433/879] 10.222.0.58 () {56 vars in 1122 bytes} [Thu Aug 25 03:01:39 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:01:39.347663 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9ed8b3-8f89-4585-b60d-3b52ada22e4a None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:39.450391 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9ed8b3-8f89-4585-b60d-3b52ada22e4a None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:39.453357 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:39.486675 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d23f952-773f-40fc-95db-da4882fcedba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:39.487307 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d23f952-773f-40fc-95db-da4882fcedba None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:39.487588 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d23f952-773f-40fc-95db-da4882fcedba None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:39.487865 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d23f952-773f-40fc-95db-da4882fcedba None admin] PATH_INFO: `/v3/services` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:39.488505 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d23f952-773f-40fc-95db-da4882fcedba None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:39.488816 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d23f952-773f-40fc-95db-da4882fcedba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:39.490192 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d23f952-773f-40fc-95db-da4882fcedba None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:39.502391 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 447/880] 10.222.0.58 () {56 vars in 1122 bytes} [Thu Aug 25 03:01:39 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:39.513518 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abfc97e-e28f-4e3f-bbe6-595d53690100 None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:39.514169 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abfc97e-e28f-4e3f-bbe6-595d53690100 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:39.514766 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abfc97e-e28f-4e3f-bbe6-595d53690100 None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:39.516725 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 434/881] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:01:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:01:39.522836 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e424780-9647-418f-bd48-00472f6152a6 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:39.523185 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e424780-9647-418f-bd48-00472f6152a6 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:39.523406 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e424780-9647-418f-bd48-00472f6152a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:39.544042 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-4e424780-9647-418f-bd48-00472f6152a6 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:01:39.545205 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-4e424780-9647-418f-bd48-00472f6152a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:01:39.619328 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abfc97e-e28f-4e3f-bbe6-595d53690100 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:39.628532 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 448/882] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:01:39 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 03:01:39.704963 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:39.736515 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abfc97e-e28f-4e3f-bbe6-595d53690100 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:39.772626 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11340324-4d3b-4533-a10c-41ef15f41758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:39.773240 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11340324-4d3b-4533-a10c-41ef15f41758 None admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:39.773517 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11340324-4d3b-4533-a10c-41ef15f41758 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:39.773793 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11340324-4d3b-4533-a10c-41ef15f41758 None admin] PATH_INFO: `/v3/services` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:39.774488 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11340324-4d3b-4533-a10c-41ef15f41758 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:39.775860 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11340324-4d3b-4533-a10c-41ef15f41758 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:39.887499 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 435/883] 10.222.0.58 () {60 vars in 1141 bytes} [Thu Aug 25 03:01:39 2022] POST /identity/v3/services => generated 269 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:01:39.900604 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e424780-9647-418f-bd48-00472f6152a6 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:40.009753 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:40.013488 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e424780-9647-418f-bd48-00472f6152a6 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:40.057443 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c56f71-7cc0-4ee4-90bf-258469ed980f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:40.058094 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c56f71-7cc0-4ee4-90bf-258469ed980f None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:40.058411 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c56f71-7cc0-4ee4-90bf-258469ed980f None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:40.058692 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c56f71-7cc0-4ee4-90bf-258469ed980f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:40.072954 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c56f71-7cc0-4ee4-90bf-258469ed980f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:01:40.078641 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-65c56f71-7cc0-4ee4-90bf-258469ed980f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 25 03:01:40.078641 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:40.078641 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:40.078641 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:40.078641 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:40.078641 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:40.078641 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:40.078641 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:40.078641 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:40.078641 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:40.078641 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:40.078641 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:40.078641 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:01:40.078641 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:01:40.078641 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:01:40.078641 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:01:40.079576 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:01:40.079576 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:01:40.079576 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:01:40.079576 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:01:40.079576 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:40.079576 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:40.079576 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:40.079576 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:40.079576 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:40.079576 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:40.079576 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:40.079576 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:40.079576 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:40.079576 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:40.079576 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:40.079576 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:40.080419 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:40.080419 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:40.080419 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:40.080419 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:40.080419 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:01:40.080419 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:01:40.080419 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:01:40.080419 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:01:40.080419 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:01:40.080419 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:01:40.080419 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 25 03:01:40.080419 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:40.080419 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 449/884] 10.222.0.58 () {56 vars in 1120 bytes} [Thu Aug 25 03:01:39 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 03:01:40.085972 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11340324-4d3b-4533-a10c-41ef15f41758 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:40.189984 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11340324-4d3b-4533-a10c-41ef15f41758 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:40.224426 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:40.230318 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a679da5-8fca-4341-b41d-ac2777885e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:40.230940 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a679da5-8fca-4341-b41d-ac2777885e57 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:40.231218 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a679da5-8fca-4341-b41d-ac2777885e57 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:40.231492 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a679da5-8fca-4341-b41d-ac2777885e57 None admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:40.232160 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a679da5-8fca-4341-b41d-ac2777885e57 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:40.232462 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a679da5-8fca-4341-b41d-ac2777885e57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:40.234550 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a679da5-8fca-4341-b41d-ac2777885e57 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:40.243646 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 436/885] 10.222.0.58 () {56 vars in 1115 bytes} [Thu Aug 25 03:01:40 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:40.248924 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c56f71-7cc0-4ee4-90bf-258469ed980f None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:40.364268 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c56f71-7cc0-4ee4-90bf-258469ed980f None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:40.398379 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d39a7b4-fc7e-4607-8d85-e34656008376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:40.399018 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d39a7b4-fc7e-4607-8d85-e34656008376 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:40.399296 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d39a7b4-fc7e-4607-8d85-e34656008376 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:40.399566 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d39a7b4-fc7e-4607-8d85-e34656008376 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:40.410632 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d39a7b4-fc7e-4607-8d85-e34656008376 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:40.412090 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d39a7b4-fc7e-4607-8d85-e34656008376 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:40.427070 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-7d39a7b4-fc7e-4607-8d85-e34656008376 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:40.427070 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:40.427070 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:40.427070 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:40.427070 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:40.427070 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:40.427070 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:40.427070 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:40.427070 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:40.427070 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:40.427070 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:40.427070 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:40.427070 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 03:01:40.427070 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 03:01:40.427070 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 03:01:40.427070 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 03:01:40.428062 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:40.428062 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:40.428062 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:40.428062 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:40.428062 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:40.428062 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:40.428062 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:40.428062 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:40.428062 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:40.428062 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:40.428062 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:40.428062 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:40.428062 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:40.428062 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:40.428062 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:40.428062 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:40.428062 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 03:01:40.429632 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 03:01:40.429632 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:40.429632 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:40.429632 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 450/886] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:01:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:40.434263 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a679da5-8fca-4341-b41d-ac2777885e57 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:40.500026 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557ee29d-89c1-42ba-a10f-b97fb93ff739 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:40.500216 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557ee29d-89c1-42ba-a10f-b97fb93ff739 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:40.500492 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557ee29d-89c1-42ba-a10f-b97fb93ff739 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:40.502083 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 451/887] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:01:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 25 03:01:40.507427 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd06ebf1-02a0-4228-90d2-4b17466f0342 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:40.507820 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd06ebf1-02a0-4228-90d2-4b17466f0342 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:40.508197 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd06ebf1-02a0-4228-90d2-4b17466f0342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:40.522670 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:40.529438 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-cd06ebf1-02a0-4228-90d2-4b17466f0342 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:01:40.531705 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-cd06ebf1-02a0-4228-90d2-4b17466f0342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:01:40.541465 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a679da5-8fca-4341-b41d-ac2777885e57 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:40.587113 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3856becd-1d53-4518-96c3-60660572222a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:40.587917 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3856becd-1d53-4518-96c3-60660572222a None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:40.588746 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3856becd-1d53-4518-96c3-60660572222a None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:40.589134 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3856becd-1d53-4518-96c3-60660572222a None admin] PATH_INFO: `/v3/domains` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:40.589959 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3856becd-1d53-4518-96c3-60660572222a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:40.590394 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3856becd-1d53-4518-96c3-60660572222a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:40.592612 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3856becd-1d53-4518-96c3-60660572222a None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:40.607553 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 437/888] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Aug 25 03:01:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:40.612921 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3856becd-1d53-4518-96c3-60660572222a None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:40.624484 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 452/889] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:01:40 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 25 03:01:40.676308 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:40.723460 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3856becd-1d53-4518-96c3-60660572222a None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:40.758067 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f86532-23c8-430f-9e17-089b227d2ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:40.759090 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f86532-23c8-430f-9e17-089b227d2ad9 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:40.759571 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f86532-23c8-430f-9e17-089b227d2ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:40.760073 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f86532-23c8-430f-9e17-089b227d2ad9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:40.768293 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f86532-23c8-430f-9e17-089b227d2ad9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:40.770846 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f86532-23c8-430f-9e17-089b227d2ad9 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:40.782159 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-86f86532-23c8-430f-9e17-089b227d2ad9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 25 03:01:40.782159 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:40.782159 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:40.782159 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:40.782159 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:40.782159 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:40.782159 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:40.782159 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:40.782159 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:40.782159 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:40.782159 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:40.782159 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:40.782159 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 25 03:01:40.782159 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 25 03:01:40.782159 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 25 03:01:40.782159 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 25 03:01:40.783127 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:40.783127 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:40.783127 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 25 03:01:40.783127 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 03:01:40.783127 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 25 03:01:40.783127 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 03:01:40.783127 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:40.783127 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:40.783127 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:40.783127 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:40.783127 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:40.783127 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:40.783127 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:40.783127 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:40.783127 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:40.783127 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:40.783127 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:40.783127 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:40.784055 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:40.784055 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:40.784055 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 25 03:01:40.784055 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 25 03:01:40.784055 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 25 03:01:40.784055 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 25 03:01:40.784055 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 25 03:01:40.784055 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 25 03:01:40.784055 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 25 03:01:40.784055 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:40.786489 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 438/890] 10.222.0.58 () {56 vars in 1117 bytes} [Thu Aug 25 03:01:40 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:40.798351 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd06ebf1-02a0-4228-90d2-4b17466f0342 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:40.897008 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd06ebf1-02a0-4228-90d2-4b17466f0342 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:40.915669 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f86532-23c8-430f-9e17-089b227d2ad9 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:40.933744 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0034d2e-2cab-4ba0-87ea-781091db395a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:40.934372 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0034d2e-2cab-4ba0-87ea-781091db395a None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:40.934660 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0034d2e-2cab-4ba0-87ea-781091db395a None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:40.934987 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0034d2e-2cab-4ba0-87ea-781091db395a None admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:40.935726 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0034d2e-2cab-4ba0-87ea-781091db395a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:40.936032 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0034d2e-2cab-4ba0-87ea-781091db395a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:40.938227 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0034d2e-2cab-4ba0-87ea-781091db395a None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:40.941183 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:40.958274 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 453/891] 10.222.0.58 () {56 vars in 1149 bytes} [Thu Aug 25 03:01:40 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:40.963858 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0034d2e-2cab-4ba0-87ea-781091db395a None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:41.024914 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f86532-23c8-430f-9e17-089b227d2ad9 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:41.039805 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:41.061235 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e70748-bdc5-4e9f-984e-eb9f1464a349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:41.062663 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e70748-bdc5-4e9f-984e-eb9f1464a349 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:41.063044 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e70748-bdc5-4e9f-984e-eb9f1464a349 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:41.063631 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e70748-bdc5-4e9f-984e-eb9f1464a349 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:41.064145 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e70748-bdc5-4e9f-984e-eb9f1464a349 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:41.065617 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e70748-bdc5-4e9f-984e-eb9f1464a349 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:41.072960 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0034d2e-2cab-4ba0-87ea-781091db395a None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:41.076949 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-35e70748-bdc5-4e9f-984e-eb9f1464a349 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 25 03:01:41.076949 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:41.076949 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 25 03:01:41.076949 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 25 03:01:41.076949 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 25 03:01:41.076949 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 25 03:01:41.076949 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 25 03:01:41.076949 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 03:01:41.076949 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 25 03:01:41.076949 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:41.076949 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 25 03:01:41.076949 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:41.076949 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:41.076949 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:41.076949 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:41.076949 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:41.076949 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:41.078821 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:41.078821 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:41.078821 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:41.078821 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:41.078821 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:41.078821 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:41.078821 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 25 03:01:41.078821 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 25 03:01:41.078821 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 25 03:01:41.078821 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 25 03:01:41.078821 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:41.078821 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:41.078821 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:41.078821 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:41.078821 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:41.078821 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:41.079669 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:41.079669 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:41.079669 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:41.079669 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:41.079669 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:41.079669 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:41.079669 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:41.079669 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:41.079669 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:41.079669 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:41.079669 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 25 03:01:41.079669 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 03:01:41.079669 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 25 03:01:41.079669 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:41.083442 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 439/892] 10.222.0.58 () {56 vars in 1114 bytes} [Thu Aug 25 03:01:40 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:41.084302 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e70748-bdc5-4e9f-984e-eb9f1464a349 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:41.115989 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079654f9-f23a-47a1-923f-3b73ac9dfd0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:41.116996 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079654f9-f23a-47a1-923f-3b73ac9dfd0b None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:41.117561 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079654f9-f23a-47a1-923f-3b73ac9dfd0b None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:41.118121 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079654f9-f23a-47a1-923f-3b73ac9dfd0b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:41.185892 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:41.194903 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079654f9-f23a-47a1-923f-3b73ac9dfd0b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:41.195056 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e70748-bdc5-4e9f-984e-eb9f1464a349 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:41.196872 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079654f9-f23a-47a1-923f-3b73ac9dfd0b None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:41.230679 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f445f06e-204d-4c3c-9e0b-85e8d53395c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:41.231368 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f445f06e-204d-4c3c-9e0b-85e8d53395c6 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:41.231647 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f445f06e-204d-4c3c-9e0b-85e8d53395c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:41.231924 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f445f06e-204d-4c3c-9e0b-85e8d53395c6 None admin] PATH_INFO: `/v3/services` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:41.232596 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f445f06e-204d-4c3c-9e0b-85e8d53395c6 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:41.232904 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f445f06e-204d-4c3c-9e0b-85e8d53395c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:41.235436 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f445f06e-204d-4c3c-9e0b-85e8d53395c6 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:41.245530 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 440/893] 10.222.0.58 () {56 vars in 1114 bytes} [Thu Aug 25 03:01:41 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:41.246394 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f445f06e-204d-4c3c-9e0b-85e8d53395c6 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:41.292519 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:41.313215 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 454/894] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Aug 25 03:01:40 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 25 03:01:41.314442 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079654f9-f23a-47a1-923f-3b73ac9dfd0b None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:41.354485 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f445f06e-204d-4c3c-9e0b-85e8d53395c6 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:41.392660 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e865bea-f4df-4f3d-8fc3-67e054f2cc13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:41.393275 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e865bea-f4df-4f3d-8fc3-67e054f2cc13 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:41.393561 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e865bea-f4df-4f3d-8fc3-67e054f2cc13 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:41.393834 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e865bea-f4df-4f3d-8fc3-67e054f2cc13 None admin] PATH_INFO: `/v3/services` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:41.394500 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e865bea-f4df-4f3d-8fc3-67e054f2cc13 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:41.394804 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e865bea-f4df-4f3d-8fc3-67e054f2cc13 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:41.396309 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e865bea-f4df-4f3d-8fc3-67e054f2cc13 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:41.406271 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 441/895] 10.222.0.58 () {56 vars in 1114 bytes} [Thu Aug 25 03:01:41 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:41.407907 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e865bea-f4df-4f3d-8fc3-67e054f2cc13 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:41.418128 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:41.425684 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079654f9-f23a-47a1-923f-3b73ac9dfd0b None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:41.463719 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b234741c-c31e-420e-a901-743b2eb2f54c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:41.464712 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b234741c-c31e-420e-a901-743b2eb2f54c None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:41.465298 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b234741c-c31e-420e-a901-743b2eb2f54c None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:41.465775 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b234741c-c31e-420e-a901-743b2eb2f54c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:41.477350 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b234741c-c31e-420e-a901-743b2eb2f54c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:41.479301 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b234741c-c31e-420e-a901-743b2eb2f54c None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:41.495617 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-b234741c-c31e-420e-a901-743b2eb2f54c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:41.495617 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:41.495617 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:41.495617 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:41.495617 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:41.495617 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:41.495617 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:41.495617 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:41.495617 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:41.495617 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:41.495617 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:41.495617 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:41.495617 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 03:01:41.495617 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 03:01:41.495617 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 03:01:41.495617 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 03:01:41.496600 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:41.496600 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:41.496600 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:41.496600 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:41.496600 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:41.496600 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:41.496600 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:41.496600 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:41.496600 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:41.496600 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:41.496600 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:41.496600 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:41.496600 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:41.496600 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:41.496600 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:41.496600 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:41.496600 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 03:01:41.497481 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 03:01:41.497481 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:41.497481 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:41.504872 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 455/896] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:01:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:41.506086 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b234741c-c31e-420e-a901-743b2eb2f54c None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:41.515770 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e865bea-f4df-4f3d-8fc3-67e054f2cc13 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:41.524013 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:41.556945 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed16a4f2-f5e1-4c52-a18d-411f910cccd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:41.557480 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed16a4f2-f5e1-4c52-a18d-411f910cccd0 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:41.557760 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed16a4f2-f5e1-4c52-a18d-411f910cccd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:41.558032 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed16a4f2-f5e1-4c52-a18d-411f910cccd0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:41.558809 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed16a4f2-f5e1-4c52-a18d-411f910cccd0 None admin] RBAC: Adding query filter params (interface=public, service_id=1ff45e2be1754b33adf36c23c9cd8447, region_id=RegionOne) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:41.559321 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed16a4f2-f5e1-4c52-a18d-411f910cccd0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:41.561245 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed16a4f2-f5e1-4c52-a18d-411f910cccd0 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:41.566762 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 442/897] 10.222.0.58 () {56 vars in 1251 bytes} [Thu Aug 25 03:01:41 2022] GET /identity/v3/endpoints?service_id=1ff45e2be1754b33adf36c23c9cd8447&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:01:41.568044 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e15ea5d-31cb-42ff-b7d7-674d11e999cb None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:41.568403 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e15ea5d-31cb-42ff-b7d7-674d11e999cb None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:41.568764 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e15ea5d-31cb-42ff-b7d7-674d11e999cb None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:41.570179 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 443/898] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:01:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:01:41.574936 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e15ea5d-31cb-42ff-b7d7-674d11e999cb None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:41.627668 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b234741c-c31e-420e-a901-743b2eb2f54c None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:41.665339 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa8fd34-0309-49cb-8b39-4449dd711371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:41.665957 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa8fd34-0309-49cb-8b39-4449dd711371 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:41.666235 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa8fd34-0309-49cb-8b39-4449dd711371 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:41.666536 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa8fd34-0309-49cb-8b39-4449dd711371 None admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:41.667214 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa8fd34-0309-49cb-8b39-4449dd711371 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:41.667516 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa8fd34-0309-49cb-8b39-4449dd711371 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:41.668877 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa8fd34-0309-49cb-8b39-4449dd711371 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:41.684885 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 456/899] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Aug 25 03:01:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:41.684885 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3219041f-ec62-4aee-a872-6f66dad67790 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:41.684885 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3219041f-ec62-4aee-a872-6f66dad67790 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:41.684885 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3219041f-ec62-4aee-a872-6f66dad67790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:41.693910 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e15ea5d-31cb-42ff-b7d7-674d11e999cb None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:41.694968 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:41.700319 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-3219041f-ec62-4aee-a872-6f66dad67790 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:01:41.701494 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-3219041f-ec62-4aee-a872-6f66dad67790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:01:41.732910 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6db725-eb4e-46a8-b08f-165382ade3c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:41.734075 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6db725-eb4e-46a8-b08f-165382ade3c0 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:41.734385 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6db725-eb4e-46a8-b08f-165382ade3c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:41.734666 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6db725-eb4e-46a8-b08f-165382ade3c0 None admin] PATH_INFO: `/v3/services` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:41.735276 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6db725-eb4e-46a8-b08f-165382ade3c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:41.736989 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6db725-eb4e-46a8-b08f-165382ade3c0 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:41.750315 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 444/900] 10.222.0.58 () {56 vars in 1087 bytes} [Thu Aug 25 03:01:41 2022] GET /identity/v3/services => generated 1268 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:01:41.750315 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6db725-eb4e-46a8-b08f-165382ade3c0 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:41.790858 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:41.791556 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 457/901] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:01:41 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 03:01:41.863170 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6db725-eb4e-46a8-b08f-165382ade3c0 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:41.903887 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e078789-d11a-4c15-a9ba-1c0e2276f4da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:41.904532 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e078789-d11a-4c15-a9ba-1c0e2276f4da None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:41.904816 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e078789-d11a-4c15-a9ba-1c0e2276f4da None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:41.905095 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e078789-d11a-4c15-a9ba-1c0e2276f4da None admin] PATH_INFO: `/v3/projects/service` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:41.913688 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e078789-d11a-4c15-a9ba-1c0e2276f4da None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:41.915245 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e078789-d11a-4c15-a9ba-1c0e2276f4da None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:41.927613 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-9e078789-d11a-4c15-a9ba-1c0e2276f4da None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 03:01:41.927613 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:41.927613 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:41.927613 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:41.927613 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:41.927613 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:41.927613 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:41.927613 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:41.927613 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:41.927613 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:41.927613 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:41.927613 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:41.927613 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 25 03:01:41.927613 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 25 03:01:41.927613 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 25 03:01:41.928550 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 25 03:01:41.928550 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:41.928550 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:41.928550 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:41.928550 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:41.928550 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:41.928550 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:41.928550 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:41.928550 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:41.928550 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:41.928550 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:41.928550 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:41.928550 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:41.928550 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:41.928550 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:41.928550 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:41.928550 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:41.929436 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 25 03:01:41.929436 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 25 03:01:41.929436 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 25 03:01:41.929436 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 25 03:01:41.929436 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 25 03:01:41.929436 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 25 03:01:41.929436 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 03:01:41.929436 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:41.929436 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 445/902] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Aug 25 03:01:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:41.936838 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3219041f-ec62-4aee-a872-6f66dad67790 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:41.990917 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eea37d-8710-4b86-a374-a16fea665281 None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:41.991236 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eea37d-8710-4b86-a374-a16fea665281 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:41.991536 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eea37d-8710-4b86-a374-a16fea665281 None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:41.994462 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 446/903] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:01:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 25 03:01:41.999228 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e08cfca-4130-489b-b0eb-1e209aa7a3fb None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:41.999516 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e08cfca-4130-489b-b0eb-1e209aa7a3fb None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:41.999785 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e08cfca-4130-489b-b0eb-1e209aa7a3fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:42.018600 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-1e08cfca-4130-489b-b0eb-1e209aa7a3fb None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:01:42.019797 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-1e08cfca-4130-489b-b0eb-1e209aa7a3fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:01:42.052756 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3219041f-ec62-4aee-a872-6f66dad67790 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:42.092483 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 447/904] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:01:41 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 03:01:42.093892 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e08cfca-4130-489b-b0eb-1e209aa7a3fb None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:42.101088 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea81aed-383e-4511-8e4f-b425b0b0a85c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:42.101704 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea81aed-383e-4511-8e4f-b425b0b0a85c None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:42.101983 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea81aed-383e-4511-8e4f-b425b0b0a85c None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:42.102254 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea81aed-383e-4511-8e4f-b425b0b0a85c None admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:42.103061 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea81aed-383e-4511-8e4f-b425b0b0a85c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:42.103365 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea81aed-383e-4511-8e4f-b425b0b0a85c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:42.104804 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea81aed-383e-4511-8e4f-b425b0b0a85c None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:42.107980 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:42.122737 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 458/905] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Aug 25 03:01:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:42.124933 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea81aed-383e-4511-8e4f-b425b0b0a85c None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:42.126820 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:42.221786 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e08cfca-4130-489b-b0eb-1e209aa7a3fb None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:42.257830 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb1a575-8209-407d-9421-26ddaaf0a765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:42.258486 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb1a575-8209-407d-9421-26ddaaf0a765 None admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:42.258768 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb1a575-8209-407d-9421-26ddaaf0a765 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:42.259043 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb1a575-8209-407d-9421-26ddaaf0a765 None admin] PATH_INFO: `/v3/services` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:42.259692 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb1a575-8209-407d-9421-26ddaaf0a765 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:42.261638 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb1a575-8209-407d-9421-26ddaaf0a765 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:42.268477 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea81aed-383e-4511-8e4f-b425b0b0a85c None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:42.304404 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4179ee-ec01-4faf-a952-f93167314ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:42.305041 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4179ee-ec01-4faf-a952-f93167314ec1 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:42.305324 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4179ee-ec01-4faf-a952-f93167314ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:42.305597 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4179ee-ec01-4faf-a952-f93167314ec1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:42.306358 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4179ee-ec01-4faf-a952-f93167314ec1 None admin] RBAC: Adding query filter params (role.id=b8d76e033b4b43ccbe4e580f17be8cc9, scope.project.id=9fd4be43faec4e4191e0e4ec4a4588c3, user.id=712c35b765b64919aa3eaaf02892590d) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:42.306671 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4179ee-ec01-4faf-a952-f93167314ec1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:42.308137 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4179ee-ec01-4faf-a952-f93167314ec1 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:42.314628 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 459/906] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Aug 25 03:01:42 2022] GET /identity/v3/role_assignments?user.id=712c35b765b64919aa3eaaf02892590d&scope.project.id=9fd4be43faec4e4191e0e4ec4a4588c3&role.id=b8d76e033b4b43ccbe4e580f17be8cc9 => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:01:42.329347 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 448/907] 10.222.0.58 () {60 vars in 1141 bytes} [Thu Aug 25 03:01:42 2022] POST /identity/v3/services => generated 241 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:01:42.370648 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4179ee-ec01-4faf-a952-f93167314ec1 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:42.449267 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:42.499925 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4179ee-ec01-4faf-a952-f93167314ec1 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:42.542029 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2910da8-b965-4d14-90b9-74a982f5a985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:42.542699 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2910da8-b965-4d14-90b9-74a982f5a985 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:42.542977 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2910da8-b965-4d14-90b9-74a982f5a985 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:42.543246 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2910da8-b965-4d14-90b9-74a982f5a985 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:42.543847 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2910da8-b965-4d14-90b9-74a982f5a985 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:42.545205 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2910da8-b965-4d14-90b9-74a982f5a985 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:42.553934 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-a2910da8-b965-4d14-90b9-74a982f5a985 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 25 03:01:42.553934 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:42.553934 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 25 03:01:42.553934 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 25 03:01:42.553934 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 25 03:01:42.553934 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 25 03:01:42.553934 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 25 03:01:42.553934 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 03:01:42.553934 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 25 03:01:42.553934 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:42.553934 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 25 03:01:42.553934 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:42.553934 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:42.553934 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:42.553934 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:42.553934 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:42.554920 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:42.554920 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:42.554920 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:42.554920 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:42.554920 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:42.554920 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:42.554920 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:42.554920 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 25 03:01:42.554920 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 25 03:01:42.554920 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 25 03:01:42.554920 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 25 03:01:42.554920 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:42.554920 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:42.554920 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:42.554920 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:42.554920 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:42.554920 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:42.555817 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:42.555817 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:42.555817 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:42.555817 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:42.555817 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:42.555817 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:42.555817 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:42.555817 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:42.555817 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:42.555817 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:42.555817 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 25 03:01:42.555817 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 03:01:42.555817 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 25 03:01:42.555817 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:42.555817 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 460/908] 10.222.0.58 () {56 vars in 1132 bytes} [Thu Aug 25 03:01:42 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:42.560967 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb1a575-8209-407d-9421-26ddaaf0a765 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:42.652174 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:42.669191 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb1a575-8209-407d-9421-26ddaaf0a765 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:42.705303 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37f4a71-4433-41ed-b959-e1bb38046ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:42.705844 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37f4a71-4433-41ed-b959-e1bb38046ce1 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:42.706124 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37f4a71-4433-41ed-b959-e1bb38046ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:42.706437 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37f4a71-4433-41ed-b959-e1bb38046ce1 None admin] PATH_INFO: `/v3/services` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:42.707146 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37f4a71-4433-41ed-b959-e1bb38046ce1 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:42.707443 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37f4a71-4433-41ed-b959-e1bb38046ce1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:42.708822 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37f4a71-4433-41ed-b959-e1bb38046ce1 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:42.714171 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 449/909] 10.222.0.58 () {56 vars in 1126 bytes} [Thu Aug 25 03:01:42 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:01:42.720538 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2910da8-b965-4d14-90b9-74a982f5a985 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:42.825808 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2910da8-b965-4d14-90b9-74a982f5a985 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:42.848216 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:42.863869 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88b9c81-38b9-4cac-8bc7-8660c6b71276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:42.864827 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88b9c81-38b9-4cac-8bc7-8660c6b71276 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:42.865393 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88b9c81-38b9-4cac-8bc7-8660c6b71276 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:42.865910 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88b9c81-38b9-4cac-8bc7-8660c6b71276 None admin] PATH_INFO: `/v3/services` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:42.866806 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88b9c81-38b9-4cac-8bc7-8660c6b71276 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:42.867368 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88b9c81-38b9-4cac-8bc7-8660c6b71276 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:42.869019 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88b9c81-38b9-4cac-8bc7-8660c6b71276 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:42.879344 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 461/910] 10.222.0.58 () {56 vars in 1126 bytes} [Thu Aug 25 03:01:42 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:42.882968 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37f4a71-4433-41ed-b959-e1bb38046ce1 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:42.989267 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37f4a71-4433-41ed-b959-e1bb38046ce1 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:43.024506 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346d10df-452c-4564-88d2-9025da49586c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:43.025120 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346d10df-452c-4564-88d2-9025da49586c None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:43.025435 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346d10df-452c-4564-88d2-9025da49586c None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:43.025668 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346d10df-452c-4564-88d2-9025da49586c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:43.026470 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346d10df-452c-4564-88d2-9025da49586c None admin] RBAC: Adding query filter params (interface=public, service_id=6a3da60ab1e0486ba5cccd1fabcaa5e2, region_id=RegionOne) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:43.026782 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346d10df-452c-4564-88d2-9025da49586c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:43.028222 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346d10df-452c-4564-88d2-9025da49586c None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:43.034335 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 450/911] 10.222.0.58 () {56 vars in 1251 bytes} [Thu Aug 25 03:01:42 2022] GET /identity/v3/endpoints?service_id=6a3da60ab1e0486ba5cccd1fabcaa5e2&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:43.039995 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88b9c81-38b9-4cac-8bc7-8660c6b71276 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:43.148062 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88b9c81-38b9-4cac-8bc7-8660c6b71276 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:43.182030 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac01d279-52ef-422a-890a-f176adc8a78a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:43.182667 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac01d279-52ef-422a-890a-f176adc8a78a None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:43.182947 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac01d279-52ef-422a-890a-f176adc8a78a None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:43.183225 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac01d279-52ef-422a-890a-f176adc8a78a None admin] PATH_INFO: `/v3/services` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:43.183829 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac01d279-52ef-422a-890a-f176adc8a78a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:43.185192 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac01d279-52ef-422a-890a-f176adc8a78a None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:43.192027 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 462/912] 10.222.0.58 () {56 vars in 1087 bytes} [Thu Aug 25 03:01:43 2022] GET /identity/v3/services => generated 1497 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:01:43.829676 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d3ea30-9a46-4b16-9ac7-d73c2e48a2bf None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:43.830184 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d3ea30-9a46-4b16-9ac7-d73c2e48a2bf None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:43.830260 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d3ea30-9a46-4b16-9ac7-d73c2e48a2bf None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:43.831851 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 451/913] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:01:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:01:43.837054 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30bff9b-50f5-4770-b120-ca1f1a729279 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:43.837342 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30bff9b-50f5-4770-b120-ca1f1a729279 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:43.837615 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30bff9b-50f5-4770-b120-ca1f1a729279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:43.855502 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-c30bff9b-50f5-4770-b120-ca1f1a729279 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:01:43.856686 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-c30bff9b-50f5-4770-b120-ca1f1a729279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:01:43.901673 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:43.924919 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 463/914] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:01:43 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 03:01:44.192081 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d3ea30-9a46-4b16-9ac7-d73c2e48a2bf None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:44.207189 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:44.299487 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d3ea30-9a46-4b16-9ac7-d73c2e48a2bf None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:44.337388 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ddfc0b-0542-472a-9a37-7c318f7644f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:44.338392 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ddfc0b-0542-472a-9a37-7c318f7644f5 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:44.338681 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ddfc0b-0542-472a-9a37-7c318f7644f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:44.338957 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ddfc0b-0542-472a-9a37-7c318f7644f5 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:44.339553 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ddfc0b-0542-472a-9a37-7c318f7644f5 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:44.339709 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3cad4c-1ab5-4947-920a-2bb42f70e85b None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:44.340134 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3cad4c-1ab5-4947-920a-2bb42f70e85b None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:44.340220 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3cad4c-1ab5-4947-920a-2bb42f70e85b None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:44.340936 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ddfc0b-0542-472a-9a37-7c318f7644f5 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:44.342011 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 464/915] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:01:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:01:44.347015 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef23d293-5977-4a1c-90ee-1aacd3c79006 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:44.347367 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef23d293-5977-4a1c-90ee-1aacd3c79006 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:44.347649 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef23d293-5977-4a1c-90ee-1aacd3c79006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:44.350426 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-88ddfc0b-0542-472a-9a37-7c318f7644f5 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 25 03:01:44.350426 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:44.350426 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 25 03:01:44.350426 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 25 03:01:44.350426 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 25 03:01:44.350426 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 25 03:01:44.350426 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 25 03:01:44.350426 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 03:01:44.350426 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 25 03:01:44.350426 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:44.350426 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 25 03:01:44.350426 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:44.350426 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:44.350426 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:44.350426 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:44.350426 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:44.350426 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:44.351417 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:44.351417 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:44.351417 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:44.351417 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:44.351417 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:44.351417 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:44.351417 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 25 03:01:44.351417 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 25 03:01:44.351417 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 25 03:01:44.351417 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 25 03:01:44.351417 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:44.351417 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:44.351417 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:44.351417 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:44.351417 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:44.351417 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:44.352863 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:44.352863 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:44.352863 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:44.352863 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:44.352863 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:44.352863 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:44.352863 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:44.352863 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:44.352863 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:44.352863 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:44.352863 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 25 03:01:44.352863 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 03:01:44.352863 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 25 03:01:44.352863 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:44.352863 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 452/916] 10.222.0.58 () {56 vars in 1114 bytes} [Thu Aug 25 03:01:44 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:44.357141 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ddfc0b-0542-472a-9a37-7c318f7644f5 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:44.366257 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-ef23d293-5977-4a1c-90ee-1aacd3c79006 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:01:44.367484 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-ef23d293-5977-4a1c-90ee-1aacd3c79006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:01:44.436476 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:44.441277 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 465/917] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:01:44 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 25 03:01:44.449860 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cb770c-9b44-4f3f-b885-16370572c0c5 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:44.449860 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cb770c-9b44-4f3f-b885-16370572c0c5 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:44.449860 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cb770c-9b44-4f3f-b885-16370572c0c5 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:44.449860 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 466/918] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:01:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:01:44.451427 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35315140-8c45-4a72-aec3-188100cd2385 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:44.452450 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35315140-8c45-4a72-aec3-188100cd2385 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:44.453198 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35315140-8c45-4a72-aec3-188100cd2385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:44.464096 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ddfc0b-0542-472a-9a37-7c318f7644f5 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:44.476052 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-35315140-8c45-4a72-aec3-188100cd2385 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:01:44.478053 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-35315140-8c45-4a72-aec3-188100cd2385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:01:44.500645 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1657135e-82ae-4657-be8a-7358719e7bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:44.501310 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1657135e-82ae-4657-be8a-7358719e7bdf None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:44.501593 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1657135e-82ae-4657-be8a-7358719e7bdf None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:44.501924 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1657135e-82ae-4657-be8a-7358719e7bdf None admin] PATH_INFO: `/v3/services` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:44.502589 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1657135e-82ae-4657-be8a-7358719e7bdf None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:44.502889 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1657135e-82ae-4657-be8a-7358719e7bdf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:44.504328 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1657135e-82ae-4657-be8a-7358719e7bdf None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:44.513449 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 453/919] 10.222.0.58 () {56 vars in 1114 bytes} [Thu Aug 25 03:01:44 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:44.515858 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1657135e-82ae-4657-be8a-7358719e7bdf None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:44.574198 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 467/920] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:01:44 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 03:01:44.638318 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1657135e-82ae-4657-be8a-7358719e7bdf None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:44.669794 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03f7093-2902-490f-bf6e-1f6737985213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:44.670442 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03f7093-2902-490f-bf6e-1f6737985213 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:44.670725 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03f7093-2902-490f-bf6e-1f6737985213 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:44.671076 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03f7093-2902-490f-bf6e-1f6737985213 None admin] PATH_INFO: `/v3/services` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:44.671709 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03f7093-2902-490f-bf6e-1f6737985213 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:44.672011 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03f7093-2902-490f-bf6e-1f6737985213 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:44.673485 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03f7093-2902-490f-bf6e-1f6737985213 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:44.678881 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 454/921] 10.222.0.58 () {56 vars in 1114 bytes} [Thu Aug 25 03:01:44 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:01:44.684481 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35315140-8c45-4a72-aec3-188100cd2385 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:44.711475 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03f7093-2902-490f-bf6e-1f6737985213 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:44.728304 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:44.729762 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:44.804498 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35315140-8c45-4a72-aec3-188100cd2385 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:44.820954 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03f7093-2902-490f-bf6e-1f6737985213 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:44.852227 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ca87d5-ebb4-4d5c-a440-d21c99f80ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:44.852980 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ca87d5-ebb4-4d5c-a440-d21c99f80ab8 None admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:44.853336 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ca87d5-ebb4-4d5c-a440-d21c99f80ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:44.853685 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ca87d5-ebb4-4d5c-a440-d21c99f80ab8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:44.854717 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ca87d5-ebb4-4d5c-a440-d21c99f80ab8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:44.856876 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ca87d5-ebb4-4d5c-a440-d21c99f80ab8 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:44.861098 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0d8b09-f62b-4eff-b97a-d3a7e8e53f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:44.862308 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0d8b09-f62b-4eff-b97a-d3a7e8e53f2b None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:44.862771 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0d8b09-f62b-4eff-b97a-d3a7e8e53f2b None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:44.863220 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0d8b09-f62b-4eff-b97a-d3a7e8e53f2b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:44.876360 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0d8b09-f62b-4eff-b97a-d3a7e8e53f2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:01:44.881161 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-ff0d8b09-f62b-4eff-b97a-d3a7e8e53f2b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 25 03:01:44.881161 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:44.881161 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:44.881161 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:44.881161 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:44.881161 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:44.881161 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:44.881161 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:44.881161 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:44.881161 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:44.881161 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:44.881161 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:44.881161 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:01:44.881161 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:01:44.881161 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:01:44.881161 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:01:44.887965 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:01:44.887965 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:01:44.887965 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:01:44.887965 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:01:44.887965 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:44.887965 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:44.887965 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:44.887965 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:44.887965 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:44.887965 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:44.887965 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:44.887965 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:44.887965 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:44.887965 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:44.887965 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:44.887965 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:44.888851 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:44.888851 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:44.888851 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:44.888851 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:44.888851 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:01:44.888851 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:01:44.888851 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:01:44.888851 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:01:44.888851 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:01:44.888851 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:01:44.888851 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 25 03:01:44.888851 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:44.888851 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 455/922] 10.222.0.58 () {56 vars in 1120 bytes} [Thu Aug 25 03:01:44 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:44.888851 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0d8b09-f62b-4eff-b97a-d3a7e8e53f2b None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:44.939159 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 468/923] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:01:44 2022] POST /identity/v3/endpoints => generated 346 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:01:44.939997 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ca87d5-ebb4-4d5c-a440-d21c99f80ab8 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:44.956317 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:45.002481 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0d8b09-f62b-4eff-b97a-d3a7e8e53f2b None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:45.050071 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a63eaa-9606-46d7-a4bd-620b3791f4ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:45.051380 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a63eaa-9606-46d7-a4bd-620b3791f4ed None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:45.051931 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a63eaa-9606-46d7-a4bd-620b3791f4ed None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:45.052411 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a63eaa-9606-46d7-a4bd-620b3791f4ed None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:45.053216 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a63eaa-9606-46d7-a4bd-620b3791f4ed None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:45.054163 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:45.056285 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a63eaa-9606-46d7-a4bd-620b3791f4ed None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:45.062230 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ca87d5-ebb4-4d5c-a440-d21c99f80ab8 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:45.067231 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-81a63eaa-9606-46d7-a4bd-620b3791f4ed None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 25 03:01:45.067231 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:45.067231 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 25 03:01:45.067231 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 25 03:01:45.067231 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 25 03:01:45.067231 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 25 03:01:45.067231 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 25 03:01:45.067231 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 03:01:45.067231 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 25 03:01:45.067231 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:45.067231 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 25 03:01:45.067231 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:45.067231 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:45.067231 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:45.067231 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:45.067231 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:45.067231 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:45.068375 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:45.068375 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:45.068375 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:45.068375 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:45.068375 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:45.068375 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:45.068375 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 25 03:01:45.068375 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 25 03:01:45.068375 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 25 03:01:45.068375 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 25 03:01:45.068375 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:45.068375 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:45.068375 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:45.068375 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:45.068375 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:45.068375 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:45.069299 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:45.069299 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:45.069299 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:45.069299 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:45.069299 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:45.069299 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:45.069299 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:45.069299 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:45.069299 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:45.069299 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:45.069299 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 25 03:01:45.069299 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 03:01:45.069299 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 25 03:01:45.069299 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:45.076898 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 456/924] 10.222.0.58 () {56 vars in 1126 bytes} [Thu Aug 25 03:01:44 2022] GET /identity/v3/services/object-store => generated 93 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:45.078456 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a63eaa-9606-46d7-a4bd-620b3791f4ed None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:45.101427 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ec334f-0118-4032-8124-85808f29f618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:45.102120 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ec334f-0118-4032-8124-85808f29f618 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:45.103462 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ec334f-0118-4032-8124-85808f29f618 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:45.103981 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ec334f-0118-4032-8124-85808f29f618 None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:45.104746 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ec334f-0118-4032-8124-85808f29f618 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:45.105125 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ec334f-0118-4032-8124-85808f29f618 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:45.106642 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ec334f-0118-4032-8124-85808f29f618 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:45.120734 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 469/925] 10.222.0.58 () {56 vars in 1115 bytes} [Thu Aug 25 03:01:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:45.122265 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ec334f-0118-4032-8124-85808f29f618 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:45.200451 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a63eaa-9606-46d7-a4bd-620b3791f4ed None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:45.224072 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:45.235630 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4f3eb8-7635-4a27-9db5-b4550efb9f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:45.238156 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4f3eb8-7635-4a27-9db5-b4550efb9f81 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:45.238156 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4f3eb8-7635-4a27-9db5-b4550efb9f81 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:45.238156 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4f3eb8-7635-4a27-9db5-b4550efb9f81 None admin] PATH_INFO: `/v3/services` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:45.238156 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4f3eb8-7635-4a27-9db5-b4550efb9f81 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:45.238156 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4f3eb8-7635-4a27-9db5-b4550efb9f81 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:45.239082 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4f3eb8-7635-4a27-9db5-b4550efb9f81 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:45.248844 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 457/926] 10.222.0.58 () {56 vars in 1122 bytes} [Thu Aug 25 03:01:45 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:45.248844 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bce054-fc09-4ded-9407-d4caa42477d3 None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:45.248844 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bce054-fc09-4ded-9407-d4caa42477d3 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:45.248844 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bce054-fc09-4ded-9407-d4caa42477d3 None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:45.248844 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 458/927] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:01:45 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:01:45.250149 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bce054-fc09-4ded-9407-d4caa42477d3 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:45.256132 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ec334f-0118-4032-8124-85808f29f618 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:45.293295 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325e8082-e590-4a7a-b72d-83ace08ef895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:45.294190 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325e8082-e590-4a7a-b72d-83ace08ef895 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:45.294656 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325e8082-e590-4a7a-b72d-83ace08ef895 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:45.295095 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325e8082-e590-4a7a-b72d-83ace08ef895 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:45.307714 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325e8082-e590-4a7a-b72d-83ace08ef895 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:45.309517 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325e8082-e590-4a7a-b72d-83ace08ef895 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:45.324883 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-325e8082-e590-4a7a-b72d-83ace08ef895 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:45.324883 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:45.324883 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:45.324883 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:45.324883 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:45.324883 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:45.324883 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:45.324883 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:45.324883 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:45.324883 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:45.324883 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:45.324883 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:45.324883 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 03:01:45.324883 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 03:01:45.324883 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 03:01:45.324883 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 03:01:45.326104 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:45.326104 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:45.326104 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:45.326104 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:45.326104 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:45.326104 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:45.326104 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:45.326104 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:45.326104 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:45.326104 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:45.326104 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:45.326104 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:45.326104 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:45.326104 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:45.326104 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:45.326104 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:45.326104 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 03:01:45.327203 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 03:01:45.327203 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:45.327203 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:45.334845 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 470/928] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:01:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:45.336079 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c63bc96-7635-4a14-95a6-bac7d9f1fc5e None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:45.336363 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c63bc96-7635-4a14-95a6-bac7d9f1fc5e None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:45.336637 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c63bc96-7635-4a14-95a6-bac7d9f1fc5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:45.355327 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-2c63bc96-7635-4a14-95a6-bac7d9f1fc5e None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:01:45.356495 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-2c63bc96-7635-4a14-95a6-bac7d9f1fc5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:01:45.359398 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:45.364410 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bce054-fc09-4ded-9407-d4caa42477d3 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:45.404276 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b496bfd-2851-48ad-b41e-25d8e3f42d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:45.404945 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b496bfd-2851-48ad-b41e-25d8e3f42d32 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:45.405757 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b496bfd-2851-48ad-b41e-25d8e3f42d32 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:45.406042 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b496bfd-2851-48ad-b41e-25d8e3f42d32 None admin] PATH_INFO: `/v3/services` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:45.406757 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b496bfd-2851-48ad-b41e-25d8e3f42d32 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:45.407063 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b496bfd-2851-48ad-b41e-25d8e3f42d32 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:45.408447 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b496bfd-2851-48ad-b41e-25d8e3f42d32 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:45.413746 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 459/929] 10.222.0.58 () {56 vars in 1122 bytes} [Thu Aug 25 03:01:45 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:45.419168 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b496bfd-2851-48ad-b41e-25d8e3f42d32 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:45.429348 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 471/930] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:01:45 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 03:01:45.432291 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c63bc96-7635-4a14-95a6-bac7d9f1fc5e None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:45.479798 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:45.557489 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b496bfd-2851-48ad-b41e-25d8e3f42d32 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:45.566876 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c63bc96-7635-4a14-95a6-bac7d9f1fc5e None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:45.619036 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbd79cb-9a94-4932-a8a3-1f3707d83781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:45.619685 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbd79cb-9a94-4932-a8a3-1f3707d83781 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:45.619966 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbd79cb-9a94-4932-a8a3-1f3707d83781 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:45.620239 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbd79cb-9a94-4932-a8a3-1f3707d83781 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:45.620954 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbd79cb-9a94-4932-a8a3-1f3707d83781 None admin] RBAC: Adding query filter params (interface=public, service_id=d1ad6cff0208431996519da505864a8c, region_id=RegionOne) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:45.621263 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbd79cb-9a94-4932-a8a3-1f3707d83781 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:45.621702 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a80e08c-9171-4b58-92c9-7f05f4e76855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:45.622607 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a80e08c-9171-4b58-92c9-7f05f4e76855 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:45.622729 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbd79cb-9a94-4932-a8a3-1f3707d83781 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:45.623260 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a80e08c-9171-4b58-92c9-7f05f4e76855 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:45.623719 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a80e08c-9171-4b58-92c9-7f05f4e76855 None admin] PATH_INFO: `/v3/domains` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:45.624574 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a80e08c-9171-4b58-92c9-7f05f4e76855 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:45.625303 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a80e08c-9171-4b58-92c9-7f05f4e76855 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:45.626934 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a80e08c-9171-4b58-92c9-7f05f4e76855 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:45.635867 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 472/931] 10.222.0.58 () {56 vars in 1251 bytes} [Thu Aug 25 03:01:45 2022] GET /identity/v3/endpoints?service_id=d1ad6cff0208431996519da505864a8c&interface=public®ion_id=RegionOne => generated 194 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:45.646319 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbd79cb-9a94-4932-a8a3-1f3707d83781 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:45.646319 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:45.646574 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 460/932] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Aug 25 03:01:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:45.648721 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a80e08c-9171-4b58-92c9-7f05f4e76855 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:45.756117 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a80e08c-9171-4b58-92c9-7f05f4e76855 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:45.759541 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbd79cb-9a94-4932-a8a3-1f3707d83781 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:45.771707 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:45.799329 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db34b378-b22f-4057-8263-a3aae164a4d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:45.800593 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db34b378-b22f-4057-8263-a3aae164a4d8 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:45.800873 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db34b378-b22f-4057-8263-a3aae164a4d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:45.801139 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db34b378-b22f-4057-8263-a3aae164a4d8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:45.801716 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13af0970-49b6-4b66-b5da-356fc4d7d42b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:45.802348 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13af0970-49b6-4b66-b5da-356fc4d7d42b None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:45.802629 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13af0970-49b6-4b66-b5da-356fc4d7d42b None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:45.802902 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13af0970-49b6-4b66-b5da-356fc4d7d42b None admin] PATH_INFO: `/v3/services` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:45.803505 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13af0970-49b6-4b66-b5da-356fc4d7d42b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:45.804901 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13af0970-49b6-4b66-b5da-356fc4d7d42b None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:45.810346 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db34b378-b22f-4057-8263-a3aae164a4d8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:45.810843 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 473/933] 10.222.0.58 () {56 vars in 1087 bytes} [Thu Aug 25 03:01:45 2022] GET /identity/v3/services => generated 1497 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:01:45.811713 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13af0970-49b6-4b66-b5da-356fc4d7d42b None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:45.812452 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db34b378-b22f-4057-8263-a3aae164a4d8 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:45.832197 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-db34b378-b22f-4057-8263-a3aae164a4d8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 25 03:01:45.832197 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:45.832197 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:45.832197 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:45.832197 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:45.832197 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:45.832197 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:45.832197 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:45.832197 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:45.832197 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:45.832197 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:45.832197 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:45.832197 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 25 03:01:45.832197 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 25 03:01:45.832197 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 25 03:01:45.832197 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 25 03:01:45.833229 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:45.833229 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:45.833229 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 25 03:01:45.833229 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 03:01:45.833229 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 25 03:01:45.833229 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 03:01:45.833229 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:45.833229 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:45.833229 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:45.833229 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:45.833229 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:45.833229 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:45.833229 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:45.833229 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:45.833229 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:45.833229 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:45.833229 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:45.833229 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:45.834172 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:45.834172 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:45.834172 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 25 03:01:45.834172 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 25 03:01:45.834172 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 25 03:01:45.834172 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 25 03:01:45.834172 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 25 03:01:45.834172 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 25 03:01:45.834172 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 25 03:01:45.834172 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:45.836640 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 461/934] 10.222.0.58 () {56 vars in 1117 bytes} [Thu Aug 25 03:01:45 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:45.843981 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db34b378-b22f-4057-8263-a3aae164a4d8 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:45.891247 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:45.933036 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13af0970-49b6-4b66-b5da-356fc4d7d42b None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:45.972995 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db34b378-b22f-4057-8263-a3aae164a4d8 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:45.973963 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d05aac-d5de-42ff-927b-a7ca9e76bd8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:45.975092 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d05aac-d5de-42ff-927b-a7ca9e76bd8b None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:45.975669 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d05aac-d5de-42ff-927b-a7ca9e76bd8b None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:45.976163 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d05aac-d5de-42ff-927b-a7ca9e76bd8b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:45.976989 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d05aac-d5de-42ff-927b-a7ca9e76bd8b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:45.978622 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d05aac-d5de-42ff-927b-a7ca9e76bd8b None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:45.989532 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-38d05aac-d5de-42ff-927b-a7ca9e76bd8b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 25 03:01:45.989532 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:45.989532 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 25 03:01:45.989532 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 25 03:01:45.989532 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 25 03:01:45.989532 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 25 03:01:45.989532 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 25 03:01:45.989532 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 03:01:45.989532 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 25 03:01:45.989532 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:45.989532 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 25 03:01:45.989532 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:45.989532 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:45.989532 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:45.989532 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:45.989532 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:45.990575 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:45.990575 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:45.990575 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:45.990575 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:45.990575 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:45.990575 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:45.990575 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:45.990575 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 25 03:01:45.990575 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 25 03:01:45.990575 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 25 03:01:45.990575 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 25 03:01:45.990575 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:45.990575 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:45.990575 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:45.990575 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:45.990575 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:45.990575 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:45.991445 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:45.991445 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:45.991445 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:45.991445 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:45.991445 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:45.991445 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:45.991445 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:45.991445 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:45.991445 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:45.991445 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:45.991445 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 25 03:01:45.991445 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 03:01:45.991445 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 25 03:01:45.991445 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:45.994323 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 474/935] 10.222.0.58 () {56 vars in 1132 bytes} [Thu Aug 25 03:01:45 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:46.002058 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d05aac-d5de-42ff-927b-a7ca9e76bd8b None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:46.017030 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a3ca6c-3122-45a1-bc6f-d3eebcf3cd6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:46.018830 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a3ca6c-3122-45a1-bc6f-d3eebcf3cd6a None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:46.018830 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a3ca6c-3122-45a1-bc6f-d3eebcf3cd6a None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:46.018830 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a3ca6c-3122-45a1-bc6f-d3eebcf3cd6a None admin] PATH_INFO: `/v3/users` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:46.019072 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a3ca6c-3122-45a1-bc6f-d3eebcf3cd6a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:46.019329 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a3ca6c-3122-45a1-bc6f-d3eebcf3cd6a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:46.020745 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a3ca6c-3122-45a1-bc6f-d3eebcf3cd6a None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:46.041886 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 462/936] 10.222.0.58 () {56 vars in 1149 bytes} [Thu Aug 25 03:01:45 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:01:46.047532 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a3ca6c-3122-45a1-bc6f-d3eebcf3cd6a None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:46.094876 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:46.136466 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d05aac-d5de-42ff-927b-a7ca9e76bd8b None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:46.168906 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:46.175646 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1b6369-d570-4e3b-ab2e-30c966732879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:46.179301 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1b6369-d570-4e3b-ab2e-30c966732879 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:46.179301 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1b6369-d570-4e3b-ab2e-30c966732879 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:46.179301 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1b6369-d570-4e3b-ab2e-30c966732879 None admin] PATH_INFO: `/v3/services` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:46.179301 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1b6369-d570-4e3b-ab2e-30c966732879 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:46.179301 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1b6369-d570-4e3b-ab2e-30c966732879 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:46.179990 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1b6369-d570-4e3b-ab2e-30c966732879 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:46.190123 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 475/937] 10.222.0.58 () {56 vars in 1126 bytes} [Thu Aug 25 03:01:46 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:46.191157 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1b6369-d570-4e3b-ab2e-30c966732879 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:46.198096 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a3ca6c-3122-45a1-bc6f-d3eebcf3cd6a None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:46.239230 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d530738-eb72-4f26-bf3b-f2a7e6759218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:46.240231 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d530738-eb72-4f26-bf3b-f2a7e6759218 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:46.240746 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d530738-eb72-4f26-bf3b-f2a7e6759218 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:46.241177 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d530738-eb72-4f26-bf3b-f2a7e6759218 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:46.252849 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d530738-eb72-4f26-bf3b-f2a7e6759218 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:46.254680 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d530738-eb72-4f26-bf3b-f2a7e6759218 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:46.270680 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-6d530738-eb72-4f26-bf3b-f2a7e6759218 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:46.270680 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:46.270680 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:46.270680 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:46.270680 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:46.270680 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:46.270680 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:46.270680 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:46.270680 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:46.270680 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:46.270680 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:46.270680 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:46.270680 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 03:01:46.270680 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 03:01:46.270680 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 03:01:46.270680 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 03:01:46.271871 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:46.271871 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:46.271871 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:46.271871 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:46.271871 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:46.271871 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:46.271871 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:46.271871 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:46.271871 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:46.271871 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:46.271871 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:46.271871 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:46.271871 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:46.271871 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:46.271871 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:46.271871 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:46.271871 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 03:01:46.272828 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 03:01:46.272828 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:46.272828 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:46.279016 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 463/938] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:01:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 03:01:46.280041 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d530738-eb72-4f26-bf3b-f2a7e6759218 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:46.301117 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1b6369-d570-4e3b-ab2e-30c966732879 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:46.338073 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e34e22-be7a-4897-a201-e7504f5cc3ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:46.338820 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e34e22-be7a-4897-a201-e7504f5cc3ce None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:46.339173 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e34e22-be7a-4897-a201-e7504f5cc3ce None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:46.339598 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e34e22-be7a-4897-a201-e7504f5cc3ce None admin] PATH_INFO: `/v3/services` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:46.340206 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e34e22-be7a-4897-a201-e7504f5cc3ce None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:46.340584 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e34e22-be7a-4897-a201-e7504f5cc3ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:46.342082 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e34e22-be7a-4897-a201-e7504f5cc3ce None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:46.351945 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 476/939] 10.222.0.58 () {56 vars in 1126 bytes} [Thu Aug 25 03:01:46 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:46.353867 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e34e22-be7a-4897-a201-e7504f5cc3ce None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:46.368443 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:46.391166 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d530738-eb72-4f26-bf3b-f2a7e6759218 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:46.426547 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:46.440213 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91a26b4-01db-497f-98b7-45bb84df7ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:46.441135 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91a26b4-01db-497f-98b7-45bb84df7ce4 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:46.445886 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91a26b4-01db-497f-98b7-45bb84df7ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:46.445886 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91a26b4-01db-497f-98b7-45bb84df7ce4 None admin] PATH_INFO: `/v3/domains` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:46.445886 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91a26b4-01db-497f-98b7-45bb84df7ce4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:46.445886 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91a26b4-01db-497f-98b7-45bb84df7ce4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:46.445886 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91a26b4-01db-497f-98b7-45bb84df7ce4 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:46.460534 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 464/940] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Aug 25 03:01:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:46.466071 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91a26b4-01db-497f-98b7-45bb84df7ce4 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:46.480826 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e34e22-be7a-4897-a201-e7504f5cc3ce None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:46.525460 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1638945-9581-4246-9550-30c4a253b3c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:46.526068 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1638945-9581-4246-9550-30c4a253b3c1 None admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:46.526375 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1638945-9581-4246-9550-30c4a253b3c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:46.526645 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1638945-9581-4246-9550-30c4a253b3c1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:46.527557 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1638945-9581-4246-9550-30c4a253b3c1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:46.529569 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1638945-9581-4246-9550-30c4a253b3c1 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:46.591277 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91a26b4-01db-497f-98b7-45bb84df7ce4 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:46.632190 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff4b8a6-064f-4c27-a762-36cc4c1e0f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:46.633214 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff4b8a6-064f-4c27-a762-36cc4c1e0f6b None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:46.633703 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff4b8a6-064f-4c27-a762-36cc4c1e0f6b None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:46.634266 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff4b8a6-064f-4c27-a762-36cc4c1e0f6b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:46.635665 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 477/941] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:01:46 2022] POST /identity/v3/endpoints => generated 347 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:01:46.649026 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff4b8a6-064f-4c27-a762-36cc4c1e0f6b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:46.651170 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff4b8a6-064f-4c27-a762-36cc4c1e0f6b None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:46.665251 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-0ff4b8a6-064f-4c27-a762-36cc4c1e0f6b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 03:01:46.665251 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:46.665251 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:46.665251 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:46.665251 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:46.665251 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:46.665251 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:46.665251 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:46.665251 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:46.665251 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:46.665251 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:46.665251 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:46.665251 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 25 03:01:46.665251 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 25 03:01:46.665251 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 25 03:01:46.666322 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 25 03:01:46.666322 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:46.666322 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:46.666322 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:46.666322 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:46.666322 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:46.666322 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:46.666322 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:46.666322 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:46.666322 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:46.666322 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:46.666322 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:46.666322 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:46.666322 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:46.666322 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:46.666322 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:46.666322 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:46.667267 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 25 03:01:46.667267 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 25 03:01:46.667267 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 25 03:01:46.667267 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 25 03:01:46.667267 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 25 03:01:46.667267 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 25 03:01:46.667267 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 03:01:46.667267 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:46.669918 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 465/942] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Aug 25 03:01:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:46.676716 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1638945-9581-4246-9550-30c4a253b3c1 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:46.799241 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1638945-9581-4246-9550-30c4a253b3c1 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:46.810876 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:46.836487 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d205a24f-cfe6-4b6f-8d05-c15fcf0c7fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:46.837105 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d205a24f-cfe6-4b6f-8d05-c15fcf0c7fd4 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:46.837452 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d205a24f-cfe6-4b6f-8d05-c15fcf0c7fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:46.837730 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d205a24f-cfe6-4b6f-8d05-c15fcf0c7fd4 None admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:46.838452 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d205a24f-cfe6-4b6f-8d05-c15fcf0c7fd4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:46.838760 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d205a24f-cfe6-4b6f-8d05-c15fcf0c7fd4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:46.840188 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d205a24f-cfe6-4b6f-8d05-c15fcf0c7fd4 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:46.853240 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 478/943] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Aug 25 03:01:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:46.859167 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff4b8a6-064f-4c27-a762-36cc4c1e0f6b None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:46.862402 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:46.964360 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff4b8a6-064f-4c27-a762-36cc4c1e0f6b None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:46.964586 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb2063f-659b-4ba8-9480-85669f1f442f None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:46.964824 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb2063f-659b-4ba8-9480-85669f1f442f None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:46.965062 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb2063f-659b-4ba8-9480-85669f1f442f None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:46.966686 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 479/944] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:01:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 25 03:01:46.974760 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe3802f-fc6b-4140-94c8-0f5ba5ac3022 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:46.975157 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe3802f-fc6b-4140-94c8-0f5ba5ac3022 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:46.978675 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe3802f-fc6b-4140-94c8-0f5ba5ac3022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:46.994838 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-7fe3802f-fc6b-4140-94c8-0f5ba5ac3022 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:01:46.996368 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-7fe3802f-fc6b-4140-94c8-0f5ba5ac3022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:01:46.999841 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fdb6c5-f3c0-431e-949f-4b065f146fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:47.000486 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fdb6c5-f3c0-431e-949f-4b065f146fb4 None admin] REQUEST_METHOD: `PUT` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:47.000774 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fdb6c5-f3c0-431e-949f-4b065f146fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:47.001050 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fdb6c5-f3c0-431e-949f-4b065f146fb4 None admin] PATH_INFO: `/v3/projects/9fd4be43faec4e4191e0e4ec4a4588c3/users/712c35b765b64919aa3eaaf02892590d/roles/b8d76e033b4b43ccbe4e580f17be8cc9` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:47.003284 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fdb6c5-f3c0-431e-949f-4b065f146fb4 None admin] RBAC: Authorizing `identity:create_grant(project_id=9fd4be43faec4e4191e0e4ec4a4588c3, user_id=712c35b765b64919aa3eaaf02892590d, role_id=b8d76e033b4b43ccbe4e580f17be8cc9)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:47.004736 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fdb6c5-f3c0-431e-949f-4b065f146fb4 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:47.070754 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 480/945] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:01:46 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 25 03:01:47.085883 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 466/946] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Aug 25 03:01:46 2022] PUT /identity/v3/projects/9fd4be43faec4e4191e0e4ec4a4588c3/users/712c35b765b64919aa3eaaf02892590d/roles/b8d76e033b4b43ccbe4e580f17be8cc9 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:01:47.346010 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe3802f-fc6b-4140-94c8-0f5ba5ac3022 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:47.448625 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:47.451851 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe3802f-fc6b-4140-94c8-0f5ba5ac3022 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:47.487363 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb4f5a3-54ed-46b6-857a-a7cbc88f1e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:47.488178 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb4f5a3-54ed-46b6-857a-a7cbc88f1e93 None admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:47.488459 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb4f5a3-54ed-46b6-857a-a7cbc88f1e93 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:47.488796 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb4f5a3-54ed-46b6-857a-a7cbc88f1e93 None admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:47.489475 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb4f5a3-54ed-46b6-857a-a7cbc88f1e93 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:47.491034 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb4f5a3-54ed-46b6-857a-a7cbc88f1e93 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:47.493660 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: WARNING keystone.server.flask.common [None req-3fb4f5a3-54ed-46b6-857a-a7cbc88f1e93 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 25 03:01:47.542886 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.sql.core [None req-3fb4f5a3-54ed-46b6-857a-a7cbc88f1e93 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 25 03:01:47.542886 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 25 03:01:47.542886 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [parameters: {'id': 'a0865b3b2f9b42b485f188d0785de10b', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 25 03:01:47.542886 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81994) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 25 03:01:47.562847 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-3fb4f5a3-54ed-46b6-857a-a7cbc88f1e93 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id a0865b3b2f9b42b485f188d0785de10b.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id a0865b3b2f9b42b485f188d0785de10b. Aug 25 03:01:47.562847 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:47.562847 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 25 03:01:47.562847 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 25 03:01:47.562847 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 25 03:01:47.562847 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 25 03:01:47.562847 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 25 03:01:47.562847 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application result = self._query(query) Aug 25 03:01:47.562847 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 25 03:01:47.562847 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application conn.query(q) Aug 25 03:01:47.562847 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 25 03:01:47.562847 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 25 03:01:47.562847 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 25 03:01:47.563711 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application result.read() Aug 25 03:01:47.563711 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 25 03:01:47.563711 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 25 03:01:47.563711 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 25 03:01:47.563711 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 25 03:01:47.563711 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 25 03:01:47.563711 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 25 03:01:47.563711 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 25 03:01:47.563711 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 25 03:01:47.563711 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 25 03:01:47.563711 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:47.563711 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 25 03:01:47.563711 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:47.563711 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:47.563711 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 25 03:01:47.563711 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 25 03:01:47.563711 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 25 03:01:47.563711 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 25 03:01:47.565323 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 25 03:01:47.565323 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application next(self.gen) Aug 25 03:01:47.565323 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 25 03:01:47.565323 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application yield resource Aug 25 03:01:47.565323 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 25 03:01:47.565323 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application next(self.gen) Aug 25 03:01:47.565323 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 25 03:01:47.565323 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self.session.rollback() Aug 25 03:01:47.565323 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 25 03:01:47.565323 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self.force_reraise() Aug 25 03:01:47.565323 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 25 03:01:47.565323 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise self.value Aug 25 03:01:47.565323 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 25 03:01:47.565323 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 25 03:01:47.565323 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 25 03:01:47.565323 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application session.commit() Aug 25 03:01:47.565323 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 25 03:01:47.567487 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 25 03:01:47.567487 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 25 03:01:47.567487 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self._prepare_impl() Aug 25 03:01:47.567487 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 25 03:01:47.567487 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self.session.flush() Aug 25 03:01:47.567487 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 25 03:01:47.567487 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self._flush(objects) Aug 25 03:01:47.567487 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 25 03:01:47.567487 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 25 03:01:47.567487 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 25 03:01:47.567487 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application compat.raise_( Aug 25 03:01:47.567487 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 25 03:01:47.567487 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception Aug 25 03:01:47.567487 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 25 03:01:47.567487 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application flush_context.execute() Aug 25 03:01:47.567487 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 25 03:01:47.567487 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rec.execute(self) Aug 25 03:01:47.569682 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 25 03:01:47.569682 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 25 03:01:47.569682 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 25 03:01:47.569682 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 25 03:01:47.569682 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 25 03:01:47.569682 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 25 03:01:47.569682 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 25 03:01:47.569682 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 25 03:01:47.569682 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 25 03:01:47.569682 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 25 03:01:47.569682 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 25 03:01:47.569682 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 25 03:01:47.569682 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 25 03:01:47.569682 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 25 03:01:47.569682 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 25 03:01:47.572572 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 25 03:01:47.572572 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 25 03:01:47.572572 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception Aug 25 03:01:47.572572 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 25 03:01:47.572572 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 25 03:01:47.572572 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 25 03:01:47.572572 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 25 03:01:47.572572 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 25 03:01:47.572572 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application result = self._query(query) Aug 25 03:01:47.572572 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 25 03:01:47.572572 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application conn.query(q) Aug 25 03:01:47.572572 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 25 03:01:47.572572 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 25 03:01:47.572572 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 25 03:01:47.572572 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application result.read() Aug 25 03:01:47.572572 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 25 03:01:47.572572 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 25 03:01:47.574739 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 25 03:01:47.574739 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 25 03:01:47.574739 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 25 03:01:47.574739 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 25 03:01:47.574739 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 25 03:01:47.574739 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 25 03:01:47.574739 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 25 03:01:47.574739 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 25 03:01:47.574739 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [parameters: {'id': 'a0865b3b2f9b42b485f188d0785de10b', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 25 03:01:47.574739 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 25 03:01:47.574739 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:47.574739 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 25 03:01:47.574739 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:47.574739 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:47.574739 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 25 03:01:47.576621 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 25 03:01:47.576621 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 25 03:01:47.576621 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 25 03:01:47.576621 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 25 03:01:47.576621 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:47.576621 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 25 03:01:47.576621 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:47.576621 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:47.576621 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:47.576621 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:47.576621 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:47.576621 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:47.576621 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:47.576621 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:47.576621 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:47.576621 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:47.576621 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:47.588546 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:47.588546 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 25 03:01:47.588546 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 25 03:01:47.588546 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:47.588546 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:47.588546 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 25 03:01:47.588546 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 25 03:01:47.588546 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id a0865b3b2f9b42b485f188d0785de10b. Aug 25 03:01:47.588546 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:47.588546 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 481/947] 10.222.0.58 () {60 vars in 1140 bytes} [Thu Aug 25 03:01:47 2022] POST /identity/v3/projects => generated 262 bytes in 219 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:47.591430 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fdb6c5-f3c0-431e-949f-4b065f146fb4 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:47.659302 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:47.699588 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fdb6c5-f3c0-431e-949f-4b065f146fb4 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:47.735268 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf97b14-78be-4634-bca1-3d0be348c531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:47.736247 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf97b14-78be-4634-bca1-3d0be348c531 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:47.736754 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf97b14-78be-4634-bca1-3d0be348c531 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:47.737728 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf97b14-78be-4634-bca1-3d0be348c531 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:47.744719 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf97b14-78be-4634-bca1-3d0be348c531 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:47.746462 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf97b14-78be-4634-bca1-3d0be348c531 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:47.758190 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-ecf97b14-78be-4634-bca1-3d0be348c531 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 03:01:47.758190 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:47.758190 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:47.758190 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:47.758190 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:47.758190 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:47.758190 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:47.758190 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:47.758190 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:47.758190 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:47.758190 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:47.758190 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:47.758190 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 25 03:01:47.758190 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 25 03:01:47.758190 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 25 03:01:47.759343 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 25 03:01:47.759343 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:47.759343 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:47.759343 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:47.759343 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:47.759343 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:47.759343 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:47.759343 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:47.759343 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:47.759343 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:47.759343 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:47.759343 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:47.759343 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:47.759343 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:47.759343 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:47.759343 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:47.759343 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:47.760205 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 25 03:01:47.760205 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 25 03:01:47.760205 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 25 03:01:47.760205 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 25 03:01:47.760205 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 25 03:01:47.760205 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 25 03:01:47.760205 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 03:01:47.760205 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:47.766684 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 467/948] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Aug 25 03:01:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:47.770026 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb4f5a3-54ed-46b6-857a-a7cbc88f1e93 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:47.889325 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb4f5a3-54ed-46b6-857a-a7cbc88f1e93 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:47.891477 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc5e6ff-bd33-4ba5-a864-37a803d6b33e None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:47.891769 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc5e6ff-bd33-4ba5-a864-37a803d6b33e None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:47.892052 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc5e6ff-bd33-4ba5-a864-37a803d6b33e None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:47.893658 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 468/949] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:01:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:01:47.898541 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575244ff-d2a1-4363-9ac2-d7f986a45e40 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:47.898828 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575244ff-d2a1-4363-9ac2-d7f986a45e40 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:47.899112 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575244ff-d2a1-4363-9ac2-d7f986a45e40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:47.917374 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-575244ff-d2a1-4363-9ac2-d7f986a45e40 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:01:47.918692 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-575244ff-d2a1-4363-9ac2-d7f986a45e40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:01:47.929217 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18d3ea8-54d1-40d1-9f75-f100c633120c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:47.930469 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18d3ea8-54d1-40d1-9f75-f100c633120c None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:47.931206 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18d3ea8-54d1-40d1-9f75-f100c633120c None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:47.931684 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18d3ea8-54d1-40d1-9f75-f100c633120c None admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:47.932705 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18d3ea8-54d1-40d1-9f75-f100c633120c None admin] RBAC: Adding query filter params (name=service) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:47.933591 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18d3ea8-54d1-40d1-9f75-f100c633120c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:47.935894 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18d3ea8-54d1-40d1-9f75-f100c633120c None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:47.966688 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 482/950] 10.222.0.58 () {56 vars in 1112 bytes} [Thu Aug 25 03:01:47 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:47.995383 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 469/951] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:01:47 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 25 03:01:48.273737 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18d3ea8-54d1-40d1-9f75-f100c633120c None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:48.293870 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:48.378240 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18d3ea8-54d1-40d1-9f75-f100c633120c None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:48.417830 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df7db5a-1779-4f6b-acd1-6e0ae5c47cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:48.418490 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df7db5a-1779-4f6b-acd1-6e0ae5c47cdd None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:48.418774 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df7db5a-1779-4f6b-acd1-6e0ae5c47cdd None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:48.419046 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df7db5a-1779-4f6b-acd1-6e0ae5c47cdd None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:48.419628 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df7db5a-1779-4f6b-acd1-6e0ae5c47cdd None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:48.421058 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df7db5a-1779-4f6b-acd1-6e0ae5c47cdd None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:48.430443 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-5df7db5a-1779-4f6b-acd1-6e0ae5c47cdd None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 25 03:01:48.430443 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:48.430443 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 25 03:01:48.430443 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 25 03:01:48.430443 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 25 03:01:48.430443 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 25 03:01:48.430443 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 25 03:01:48.430443 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 03:01:48.430443 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 25 03:01:48.430443 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:48.430443 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 25 03:01:48.430443 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:48.430443 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:48.430443 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:48.430443 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:48.430443 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:48.430443 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:48.431526 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:48.431526 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:48.431526 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:48.431526 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:48.431526 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:48.431526 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:48.431526 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 25 03:01:48.431526 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 25 03:01:48.431526 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 25 03:01:48.431526 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 25 03:01:48.431526 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:48.431526 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:48.431526 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:48.431526 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:48.431526 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:48.431526 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:48.433023 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:48.433023 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:48.433023 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:48.433023 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:48.433023 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:48.433023 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:48.433023 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:48.433023 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:48.433023 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:48.433023 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:48.433023 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 25 03:01:48.433023 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 03:01:48.433023 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 25 03:01:48.433023 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:48.433023 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 483/952] 10.222.0.58 () {56 vars in 1126 bytes} [Thu Aug 25 03:01:48 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:48.437491 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575244ff-d2a1-4363-9ac2-d7f986a45e40 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:48.490873 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:48.539485 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575244ff-d2a1-4363-9ac2-d7f986a45e40 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:48.574587 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4ec1ed-af73-446c-bc74-4d45ff3b190b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:48.575447 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4ec1ed-af73-446c-bc74-4d45ff3b190b None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:48.575873 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4ec1ed-af73-446c-bc74-4d45ff3b190b None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:48.576326 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4ec1ed-af73-446c-bc74-4d45ff3b190b None admin] PATH_INFO: `/v3/services` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:48.577562 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4ec1ed-af73-446c-bc74-4d45ff3b190b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:48.578350 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4ec1ed-af73-446c-bc74-4d45ff3b190b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:48.579877 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4ec1ed-af73-446c-bc74-4d45ff3b190b None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:48.585441 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 470/953] 10.222.0.58 () {56 vars in 1122 bytes} [Thu Aug 25 03:01:48 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:48.591573 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df7db5a-1779-4f6b-acd1-6e0ae5c47cdd None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:48.651783 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b4a4e1-2f5d-475a-be2b-77bea58e62bb None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:48.652068 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b4a4e1-2f5d-475a-be2b-77bea58e62bb None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:48.652336 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b4a4e1-2f5d-475a-be2b-77bea58e62bb None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:48.653901 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 471/954] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:01:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:01:48.659807 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7157824c-efbd-4b81-b4fe-03971fffbbe5 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:48.660095 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7157824c-efbd-4b81-b4fe-03971fffbbe5 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:48.660366 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7157824c-efbd-4b81-b4fe-03971fffbbe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:48.678876 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:48.682622 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-7157824c-efbd-4b81-b4fe-03971fffbbe5 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:01:48.684071 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-7157824c-efbd-4b81-b4fe-03971fffbbe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:01:48.698862 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df7db5a-1779-4f6b-acd1-6e0ae5c47cdd None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:48.737620 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d71172-2b96-4849-aaa1-ea4949b74c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:48.738238 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d71172-2b96-4849-aaa1-ea4949b74c71 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:48.738574 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d71172-2b96-4849-aaa1-ea4949b74c71 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:48.738813 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d71172-2b96-4849-aaa1-ea4949b74c71 None admin] PATH_INFO: `/v3/services` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:48.739450 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d71172-2b96-4849-aaa1-ea4949b74c71 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:48.739752 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d71172-2b96-4849-aaa1-ea4949b74c71 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:48.741105 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d71172-2b96-4849-aaa1-ea4949b74c71 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:48.746662 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 484/955] 10.222.0.58 () {56 vars in 1122 bytes} [Thu Aug 25 03:01:48 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:48.752561 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d71172-2b96-4849-aaa1-ea4949b74c71 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:48.760450 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 472/956] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:01:48 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 03:01:48.866059 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d71172-2b96-4849-aaa1-ea4949b74c71 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:48.901445 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bd204d-e69c-4def-b82b-996f16a09e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:48.902111 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bd204d-e69c-4def-b82b-996f16a09e39 None admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:48.902478 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bd204d-e69c-4def-b82b-996f16a09e39 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:48.902759 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bd204d-e69c-4def-b82b-996f16a09e39 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:48.903425 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bd204d-e69c-4def-b82b-996f16a09e39 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:48.904788 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bd204d-e69c-4def-b82b-996f16a09e39 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:48.978822 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:48.985085 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 485/957] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:01:48 2022] POST /identity/v3/endpoints => generated 349 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:01:49.037373 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7157824c-efbd-4b81-b4fe-03971fffbbe5 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:49.090408 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:49.142662 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8d6e17-83e5-4f7a-82ad-2028a4e96401 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:49.142867 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8d6e17-83e5-4f7a-82ad-2028a4e96401 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:49.143132 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8d6e17-83e5-4f7a-82ad-2028a4e96401 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:49.144496 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 486/958] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:01:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:01:49.149868 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a126d7-867f-46b9-a853-311dfd1f8648 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:49.150152 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a126d7-867f-46b9-a853-311dfd1f8648 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:49.150462 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a126d7-867f-46b9-a853-311dfd1f8648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:49.161023 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7157824c-efbd-4b81-b4fe-03971fffbbe5 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:49.174416 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-42a126d7-867f-46b9-a853-311dfd1f8648 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:01:49.176177 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-42a126d7-867f-46b9-a853-311dfd1f8648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:01:49.218862 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026bab5c-41d2-4466-8847-7cff84594400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:49.219503 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026bab5c-41d2-4466-8847-7cff84594400 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:49.219782 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026bab5c-41d2-4466-8847-7cff84594400 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:49.220052 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026bab5c-41d2-4466-8847-7cff84594400 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:49.220632 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026bab5c-41d2-4466-8847-7cff84594400 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:49.222010 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026bab5c-41d2-4466-8847-7cff84594400 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:49.231219 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-026bab5c-41d2-4466-8847-7cff84594400 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 25 03:01:49.231219 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:49.231219 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 25 03:01:49.231219 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 25 03:01:49.231219 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 25 03:01:49.231219 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 25 03:01:49.231219 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 25 03:01:49.231219 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 03:01:49.231219 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 25 03:01:49.231219 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:49.231219 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 25 03:01:49.231219 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:49.231219 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:49.231219 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:49.231219 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:49.231219 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:49.231219 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:49.232243 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:49.232243 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:49.232243 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:49.232243 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:49.232243 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:49.232243 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:49.232243 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 25 03:01:49.232243 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 25 03:01:49.232243 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 25 03:01:49.232243 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 25 03:01:49.232243 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:49.232243 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:49.232243 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:49.232243 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:49.232243 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:49.232243 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:49.233259 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:49.233259 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:49.233259 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:49.233259 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:49.233259 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:49.233259 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:49.233259 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:49.233259 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:49.233259 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:49.233259 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:49.233259 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 25 03:01:49.233259 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 03:01:49.233259 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 25 03:01:49.233259 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:49.233259 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 473/959] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Aug 25 03:01:49 2022] GET /identity/v3/services/compute => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:49.238299 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026bab5c-41d2-4466-8847-7cff84594400 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:49.275512 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 487/960] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:01:49 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 25 03:01:49.347619 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026bab5c-41d2-4466-8847-7cff84594400 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:49.355353 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:49.385442 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79eaef4-3ad8-417e-ada9-e911e0973648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:49.386355 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79eaef4-3ad8-417e-ada9-e911e0973648 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:49.386783 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79eaef4-3ad8-417e-ada9-e911e0973648 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:49.387222 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79eaef4-3ad8-417e-ada9-e911e0973648 None admin] PATH_INFO: `/v3/services` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:49.388055 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79eaef4-3ad8-417e-ada9-e911e0973648 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:49.388502 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79eaef4-3ad8-417e-ada9-e911e0973648 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:49.390068 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79eaef4-3ad8-417e-ada9-e911e0973648 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:49.395488 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 474/961] 10.222.0.58 () {56 vars in 1112 bytes} [Thu Aug 25 03:01:49 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:49.403367 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a126d7-867f-46b9-a853-311dfd1f8648 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:49.508528 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a126d7-867f-46b9-a853-311dfd1f8648 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:49.541424 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:49.547414 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1fa793-7ac2-4685-ab57-e6b29f73a95d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:49.548025 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1fa793-7ac2-4685-ab57-e6b29f73a95d None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:49.548304 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1fa793-7ac2-4685-ab57-e6b29f73a95d None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:49.548575 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1fa793-7ac2-4685-ab57-e6b29f73a95d None admin] PATH_INFO: `/v3/services` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:49.549208 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1fa793-7ac2-4685-ab57-e6b29f73a95d None admin] RBAC: Adding query filter params (type=compute) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:49.549503 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1fa793-7ac2-4685-ab57-e6b29f73a95d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:49.550971 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1fa793-7ac2-4685-ab57-e6b29f73a95d None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:49.555830 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 488/962] 10.222.0.58 () {56 vars in 1112 bytes} [Thu Aug 25 03:01:49 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:49.556863 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79eaef4-3ad8-417e-ada9-e911e0973648 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:49.686000 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79eaef4-3ad8-417e-ada9-e911e0973648 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:49.725937 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efbd99e-f66e-488a-9064-b3f31479020b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:49.726607 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efbd99e-f66e-488a-9064-b3f31479020b None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:49.726883 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efbd99e-f66e-488a-9064-b3f31479020b None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:49.727154 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efbd99e-f66e-488a-9064-b3f31479020b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:49.737849 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efbd99e-f66e-488a-9064-b3f31479020b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:01:49.742650 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-3efbd99e-f66e-488a-9064-b3f31479020b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 25 03:01:49.742650 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:49.742650 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:49.742650 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:49.742650 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:49.742650 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:49.742650 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:49.742650 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:49.742650 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:49.742650 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:49.742650 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:49.742650 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:49.742650 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:01:49.742650 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:01:49.742650 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:01:49.742650 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:01:49.743611 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:01:49.743611 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:01:49.743611 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:01:49.743611 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:01:49.743611 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:49.743611 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:49.743611 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:49.743611 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:49.743611 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:49.743611 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:49.743611 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:49.743611 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:49.743611 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:49.743611 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:49.743611 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:49.743611 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:49.744505 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:49.744505 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:49.744505 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:49.744505 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:49.744505 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:01:49.744505 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:01:49.744505 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:01:49.744505 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:01:49.744505 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:01:49.744505 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:01:49.744505 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 25 03:01:49.744505 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:49.744505 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 475/963] 10.222.0.58 () {56 vars in 1120 bytes} [Thu Aug 25 03:01:49 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 03:01:49.752257 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1fa793-7ac2-4685-ab57-e6b29f73a95d None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:49.860324 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1fa793-7ac2-4685-ab57-e6b29f73a95d None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:49.895316 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00029a67-a6e7-42f6-99ad-4bd7bd2ecdd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:49.896293 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00029a67-a6e7-42f6-99ad-4bd7bd2ecdd8 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:49.896832 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00029a67-a6e7-42f6-99ad-4bd7bd2ecdd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:49.897307 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00029a67-a6e7-42f6-99ad-4bd7bd2ecdd8 None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:49.898196 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00029a67-a6e7-42f6-99ad-4bd7bd2ecdd8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:49.898754 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00029a67-a6e7-42f6-99ad-4bd7bd2ecdd8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:49.900341 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00029a67-a6e7-42f6-99ad-4bd7bd2ecdd8 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:49.914633 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 489/964] 10.222.0.58 () {56 vars in 1115 bytes} [Thu Aug 25 03:01:49 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:49.917334 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efbd99e-f66e-488a-9064-b3f31479020b None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:49.949894 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:50.026307 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efbd99e-f66e-488a-9064-b3f31479020b None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:50.052982 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f466fbf-7256-4b8e-8e3d-5203b5ff54fb None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:50.053286 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f466fbf-7256-4b8e-8e3d-5203b5ff54fb None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:50.053528 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f466fbf-7256-4b8e-8e3d-5203b5ff54fb None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:50.055184 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 490/965] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:01:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:01:50.060147 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0d99d0-4e5e-4200-87f9-4226a931731b None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:50.060434 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0d99d0-4e5e-4200-87f9-4226a931731b None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:50.060714 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0d99d0-4e5e-4200-87f9-4226a931731b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:50.061945 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a0ecd0-1b46-4c0b-9aa4-174f42094076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:50.062850 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a0ecd0-1b46-4c0b-9aa4-174f42094076 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:50.063356 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a0ecd0-1b46-4c0b-9aa4-174f42094076 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:50.063634 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a0ecd0-1b46-4c0b-9aa4-174f42094076 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:50.076909 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a0ecd0-1b46-4c0b-9aa4-174f42094076 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:50.078820 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a0ecd0-1b46-4c0b-9aa4-174f42094076 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:50.080703 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-da0d99d0-4e5e-4200-87f9-4226a931731b None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:01:50.082219 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-da0d99d0-4e5e-4200-87f9-4226a931731b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:01:50.099114 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-50a0ecd0-1b46-4c0b-9aa4-174f42094076 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:50.099114 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:50.099114 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:50.099114 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:50.099114 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:50.099114 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:50.099114 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:50.099114 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:50.099114 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:50.099114 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:50.099114 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:50.099114 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:50.099114 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 03:01:50.099114 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 03:01:50.099114 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 03:01:50.099114 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 03:01:50.100204 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:50.100204 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:50.100204 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:50.100204 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:50.100204 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:50.100204 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:50.100204 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:50.100204 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:50.100204 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:50.100204 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:50.100204 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:50.100204 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:50.100204 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:50.100204 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:50.100204 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:50.100204 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:50.100204 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 03:01:50.101106 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 03:01:50.101106 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:50.101106 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:50.101106 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 476/966] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:01:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:50.106101 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a0ecd0-1b46-4c0b-9aa4-174f42094076 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:50.150748 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:50.174323 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 491/967] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:01:50 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 25 03:01:50.197102 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:50.220988 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a0ecd0-1b46-4c0b-9aa4-174f42094076 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:50.258755 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e41fb2b-dfcb-4024-9407-c6b06cad819b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:50.259365 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e41fb2b-dfcb-4024-9407-c6b06cad819b None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:50.259649 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e41fb2b-dfcb-4024-9407-c6b06cad819b None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:50.259992 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e41fb2b-dfcb-4024-9407-c6b06cad819b None admin] PATH_INFO: `/v3/domains` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:50.260642 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e41fb2b-dfcb-4024-9407-c6b06cad819b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:50.260946 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e41fb2b-dfcb-4024-9407-c6b06cad819b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:50.262372 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e41fb2b-dfcb-4024-9407-c6b06cad819b None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:50.278622 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 477/968] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Aug 25 03:01:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:01:50.280846 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0d99d0-4e5e-4200-87f9-4226a931731b None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:50.388100 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0d99d0-4e5e-4200-87f9-4226a931731b None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:50.427046 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56927294-2c9b-4912-b5eb-a8d20dcbcb2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:50.427988 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56927294-2c9b-4912-b5eb-a8d20dcbcb2e None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:50.428474 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56927294-2c9b-4912-b5eb-a8d20dcbcb2e None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:50.428964 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56927294-2c9b-4912-b5eb-a8d20dcbcb2e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:50.436294 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56927294-2c9b-4912-b5eb-a8d20dcbcb2e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:50.438530 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56927294-2c9b-4912-b5eb-a8d20dcbcb2e None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:50.450406 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-56927294-2c9b-4912-b5eb-a8d20dcbcb2e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 25 03:01:50.450406 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:50.450406 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:50.450406 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:50.450406 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:50.450406 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:50.450406 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:50.450406 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:50.450406 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:50.450406 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:50.450406 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:50.450406 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:50.450406 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 25 03:01:50.450406 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 25 03:01:50.450406 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 25 03:01:50.450406 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 25 03:01:50.451371 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:50.451371 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:50.451371 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 25 03:01:50.451371 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 03:01:50.451371 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 25 03:01:50.451371 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 03:01:50.451371 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:50.451371 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:50.451371 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:50.451371 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:50.451371 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:50.451371 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:50.451371 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:50.451371 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:50.451371 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:50.451371 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:50.451371 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:50.451371 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:50.452393 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:50.452393 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:50.452393 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 25 03:01:50.452393 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 25 03:01:50.452393 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 25 03:01:50.452393 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 25 03:01:50.452393 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 25 03:01:50.452393 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 25 03:01:50.452393 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 25 03:01:50.452393 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:50.455195 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 492/969] 10.222.0.58 () {56 vars in 1117 bytes} [Thu Aug 25 03:01:50 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:50.463794 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e41fb2b-dfcb-4024-9407-c6b06cad819b None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:50.481214 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56927294-2c9b-4912-b5eb-a8d20dcbcb2e None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:50.549979 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:50.575777 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e41fb2b-dfcb-4024-9407-c6b06cad819b None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:50.619337 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8982fc21-ec04-41d7-9f68-943b7a123874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:50.620372 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8982fc21-ec04-41d7-9f68-943b7a123874 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:50.620894 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8982fc21-ec04-41d7-9f68-943b7a123874 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:50.621047 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56927294-2c9b-4912-b5eb-a8d20dcbcb2e None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:50.621803 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8982fc21-ec04-41d7-9f68-943b7a123874 None admin] PATH_INFO: `/v3/users` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:50.623674 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8982fc21-ec04-41d7-9f68-943b7a123874 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:50.624519 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8982fc21-ec04-41d7-9f68-943b7a123874 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:50.626324 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8982fc21-ec04-41d7-9f68-943b7a123874 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:50.650455 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:50.657754 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 478/970] 10.222.0.58 () {56 vars in 1149 bytes} [Thu Aug 25 03:01:50 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:50.663575 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8982fc21-ec04-41d7-9f68-943b7a123874 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:50.667137 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2705d474-3b6d-4acf-8144-30c37d26574c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:50.668613 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2705d474-3b6d-4acf-8144-30c37d26574c None admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:50.668613 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2705d474-3b6d-4acf-8144-30c37d26574c None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:50.668613 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2705d474-3b6d-4acf-8144-30c37d26574c None admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:50.669629 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2705d474-3b6d-4acf-8144-30c37d26574c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:50.672103 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2705d474-3b6d-4acf-8144-30c37d26574c None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:50.720369 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.sql.core [None req-2705d474-3b6d-4acf-8144-30c37d26574c None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 25 03:01:50.720369 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 25 03:01:50.720369 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [parameters: {'user_id': 'cb7f2fb6599b4da28ce65cbc7e522aae', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 25 03:01:50.720369 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81994) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 25 03:01:50.728961 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-2705d474-3b6d-4acf-8144-30c37d26574c None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 25 03:01:50.728961 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:50.728961 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 25 03:01:50.728961 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 25 03:01:50.728961 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 25 03:01:50.728961 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 25 03:01:50.728961 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 25 03:01:50.728961 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application result = self._query(query) Aug 25 03:01:50.728961 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 25 03:01:50.728961 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application conn.query(q) Aug 25 03:01:50.728961 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 25 03:01:50.728961 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 25 03:01:50.728961 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 25 03:01:50.728961 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application result.read() Aug 25 03:01:50.728961 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 25 03:01:50.732942 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 25 03:01:50.732942 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 25 03:01:50.732942 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 25 03:01:50.732942 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 25 03:01:50.732942 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 25 03:01:50.732942 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 25 03:01:50.732942 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 25 03:01:50.732942 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 25 03:01:50.732942 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:50.732942 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 25 03:01:50.732942 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:50.732942 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:50.732942 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 25 03:01:50.732942 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 25 03:01:50.732942 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 25 03:01:50.732942 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 25 03:01:50.732942 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 25 03:01:50.732942 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application next(self.gen) Aug 25 03:01:50.734578 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 25 03:01:50.734578 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application yield resource Aug 25 03:01:50.734578 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 25 03:01:50.734578 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application next(self.gen) Aug 25 03:01:50.734578 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 25 03:01:50.734578 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self.session.rollback() Aug 25 03:01:50.734578 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 25 03:01:50.734578 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self.force_reraise() Aug 25 03:01:50.734578 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 25 03:01:50.734578 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise self.value Aug 25 03:01:50.734578 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 25 03:01:50.734578 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 25 03:01:50.734578 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 25 03:01:50.734578 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application session.commit() Aug 25 03:01:50.734578 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 25 03:01:50.734578 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 25 03:01:50.734578 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 25 03:01:50.735487 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self._prepare_impl() Aug 25 03:01:50.735487 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 25 03:01:50.735487 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self.session.flush() Aug 25 03:01:50.735487 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 25 03:01:50.735487 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self._flush(objects) Aug 25 03:01:50.735487 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 25 03:01:50.735487 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 25 03:01:50.735487 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 25 03:01:50.735487 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application compat.raise_( Aug 25 03:01:50.735487 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 25 03:01:50.735487 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception Aug 25 03:01:50.735487 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 25 03:01:50.735487 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application flush_context.execute() Aug 25 03:01:50.735487 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 25 03:01:50.735487 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rec.execute(self) Aug 25 03:01:50.735487 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 25 03:01:50.735487 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 25 03:01:50.736892 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 25 03:01:50.736892 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 25 03:01:50.736892 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 25 03:01:50.736892 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 25 03:01:50.736892 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 25 03:01:50.736892 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 25 03:01:50.736892 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 25 03:01:50.736892 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 25 03:01:50.736892 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 25 03:01:50.736892 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 25 03:01:50.736892 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 25 03:01:50.736892 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 25 03:01:50.736892 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 25 03:01:50.736892 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 25 03:01:50.736892 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 25 03:01:50.736892 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception Aug 25 03:01:50.738691 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 25 03:01:50.738691 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 25 03:01:50.738691 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 25 03:01:50.738691 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 25 03:01:50.738691 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 25 03:01:50.738691 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application result = self._query(query) Aug 25 03:01:50.738691 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 25 03:01:50.738691 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application conn.query(q) Aug 25 03:01:50.738691 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 25 03:01:50.738691 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 25 03:01:50.738691 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 25 03:01:50.738691 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application result.read() Aug 25 03:01:50.738691 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 25 03:01:50.738691 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 25 03:01:50.738691 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 25 03:01:50.738691 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 25 03:01:50.740542 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 25 03:01:50.740542 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 25 03:01:50.740542 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 25 03:01:50.740542 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 25 03:01:50.740542 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 25 03:01:50.740542 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 25 03:01:50.740542 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [parameters: {'user_id': 'cb7f2fb6599b4da28ce65cbc7e522aae', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 25 03:01:50.740542 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 25 03:01:50.740542 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:50.740542 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 25 03:01:50.740542 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:50.740542 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:50.740542 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:50.740542 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:50.740542 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:50.742220 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:50.742220 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:50.742220 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:50.742220 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:50.742220 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:50.742220 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:50.742220 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:50.742220 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 25 03:01:50.742220 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 25 03:01:50.742220 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:50.742220 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:50.742220 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 25 03:01:50.742220 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 03:01:50.742220 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 25 03:01:50.742220 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 03:01:50.742220 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 25 03:01:50.742220 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 25 03:01:50.744911 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 25 03:01:50.744911 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 25 03:01:50.744911 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 25 03:01:50.744911 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 25 03:01:50.744911 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 25 03:01:50.744911 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:50.744911 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 493/971] 10.222.0.58 () {60 vars in 1131 bytes} [Thu Aug 25 03:01:50 2022] POST /identity/v3/users => generated 160 bytes in 250 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 25 03:01:50.744911 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2705d474-3b6d-4acf-8144-30c37d26574c None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:50.796566 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8982fc21-ec04-41d7-9f68-943b7a123874 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:50.834147 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05a3b95-96d2-4926-8c4b-827cc5d05786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:50.835119 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05a3b95-96d2-4926-8c4b-827cc5d05786 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:50.835627 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05a3b95-96d2-4926-8c4b-827cc5d05786 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:50.836082 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05a3b95-96d2-4926-8c4b-827cc5d05786 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:50.852745 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2705d474-3b6d-4acf-8144-30c37d26574c None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:50.863714 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:50.892620 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed296a6e-09de-44b7-88ab-828f0cbcaad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:50.893385 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed296a6e-09de-44b7-88ab-828f0cbcaad9 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:50.893737 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed296a6e-09de-44b7-88ab-828f0cbcaad9 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:50.894088 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed296a6e-09de-44b7-88ab-828f0cbcaad9 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:50.904736 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed296a6e-09de-44b7-88ab-828f0cbcaad9 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:50.907745 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed296a6e-09de-44b7-88ab-828f0cbcaad9 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:50.911640 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:50.914206 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05a3b95-96d2-4926-8c4b-827cc5d05786 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:50.915762 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05a3b95-96d2-4926-8c4b-827cc5d05786 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:50.922809 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-ed296a6e-09de-44b7-88ab-828f0cbcaad9 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 25 03:01:50.922809 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:50.922809 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:50.922809 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:50.922809 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:50.922809 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:50.922809 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:50.922809 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:50.922809 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:50.922809 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:50.922809 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:50.922809 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:50.922809 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 25 03:01:50.922809 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 25 03:01:50.922809 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 25 03:01:50.922809 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 25 03:01:50.923790 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:50.923790 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:50.923790 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 25 03:01:50.923790 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 03:01:50.923790 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 25 03:01:50.923790 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 03:01:50.923790 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:50.923790 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:50.923790 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:50.923790 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:50.923790 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:50.923790 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:50.923790 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:50.923790 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:50.923790 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:50.923790 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:50.923790 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:50.923790 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:50.925572 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:50.925572 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:50.925572 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 25 03:01:50.925572 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 25 03:01:50.925572 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 25 03:01:50.925572 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 25 03:01:50.925572 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 25 03:01:50.925572 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 25 03:01:50.925572 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 25 03:01:50.925572 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:50.927579 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 494/972] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Aug 25 03:01:50 2022] GET /identity/v3/users/cinder => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:50.934328 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed296a6e-09de-44b7-88ab-828f0cbcaad9 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:51.025711 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 479/973] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Aug 25 03:01:50 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 25 03:01:51.031412 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05a3b95-96d2-4926-8c4b-827cc5d05786 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:51.042867 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed296a6e-09de-44b7-88ab-828f0cbcaad9 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:51.079645 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9667369-4382-40b1-9cd3-c14033127dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:51.080688 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9667369-4382-40b1-9cd3-c14033127dc4 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:51.081114 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9667369-4382-40b1-9cd3-c14033127dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:51.081654 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9667369-4382-40b1-9cd3-c14033127dc4 None admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:51.082593 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9667369-4382-40b1-9cd3-c14033127dc4 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:51.083079 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9667369-4382-40b1-9cd3-c14033127dc4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:51.084834 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9667369-4382-40b1-9cd3-c14033127dc4 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:51.134541 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 495/974] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Aug 25 03:01:50 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:51.135594 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:51.138544 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05a3b95-96d2-4926-8c4b-827cc5d05786 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:51.146443 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd94af85-9efc-461e-b011-0d13901e60a5 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:51.146735 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd94af85-9efc-461e-b011-0d13901e60a5 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:51.147018 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd94af85-9efc-461e-b011-0d13901e60a5 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:51.152733 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 496/975] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:01:51 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 25 03:01:51.154306 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaaeeed-d439-4788-94b7-c6fbacbea981 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:51.154593 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaaeeed-d439-4788-94b7-c6fbacbea981 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:51.154872 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaaeeed-d439-4788-94b7-c6fbacbea981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:51.177945 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-feaaeeed-d439-4788-94b7-c6fbacbea981 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:01:51.179772 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e007e71e-f679-4879-88d8-5223ca94a305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:51.179919 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-feaaeeed-d439-4788-94b7-c6fbacbea981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:01:51.180427 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e007e71e-f679-4879-88d8-5223ca94a305 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:51.180717 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e007e71e-f679-4879-88d8-5223ca94a305 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:51.180999 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e007e71e-f679-4879-88d8-5223ca94a305 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:51.193667 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e007e71e-f679-4879-88d8-5223ca94a305 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:51.195318 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e007e71e-f679-4879-88d8-5223ca94a305 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:51.206430 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:51.215489 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-e007e71e-f679-4879-88d8-5223ca94a305 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:51.215489 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:51.215489 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:51.215489 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:51.215489 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:51.215489 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:51.215489 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:51.215489 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:51.215489 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:51.215489 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:51.215489 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:51.215489 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:51.215489 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 03:01:51.215489 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 03:01:51.215489 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 03:01:51.215489 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 03:01:51.225330 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:51.225330 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:51.225330 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:51.225330 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:51.225330 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:51.225330 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:51.225330 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:51.225330 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:51.225330 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:51.225330 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:51.225330 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:51.225330 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:51.225330 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:51.225330 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:51.225330 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:51.225330 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:51.225330 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 03:01:51.226571 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 03:01:51.226571 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:01:51.226571 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:51.226571 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 480/976] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:01:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:51.226571 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e007e71e-f679-4879-88d8-5223ca94a305 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:51.275152 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 497/977] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:01:51 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 03:01:51.351004 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e007e71e-f679-4879-88d8-5223ca94a305 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:51.388953 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2fb5b9-2e9e-4c04-96d3-c38c8fd8f69e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:51.389614 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2fb5b9-2e9e-4c04-96d3-c38c8fd8f69e None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:51.390480 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2fb5b9-2e9e-4c04-96d3-c38c8fd8f69e None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:51.390758 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2fb5b9-2e9e-4c04-96d3-c38c8fd8f69e None admin] PATH_INFO: `/v3/domains` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:51.391472 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2fb5b9-2e9e-4c04-96d3-c38c8fd8f69e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:51.391818 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2fb5b9-2e9e-4c04-96d3-c38c8fd8f69e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:51.393573 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2fb5b9-2e9e-4c04-96d3-c38c8fd8f69e None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:51.406131 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 481/978] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Aug 25 03:01:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:01:51.411829 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaaeeed-d439-4788-94b7-c6fbacbea981 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:51.512913 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaaeeed-d439-4788-94b7-c6fbacbea981 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:51.545515 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2fb5b9-2e9e-4c04-96d3-c38c8fd8f69e None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:51.547425 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0073c2-b9d2-41f3-883e-129a9b709966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:51.548061 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0073c2-b9d2-41f3-883e-129a9b709966 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:51.548344 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0073c2-b9d2-41f3-883e-129a9b709966 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:51.548615 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0073c2-b9d2-41f3-883e-129a9b709966 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:51.555725 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0073c2-b9d2-41f3-883e-129a9b709966 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:51.557163 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0073c2-b9d2-41f3-883e-129a9b709966 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:51.560484 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:51.568908 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-4f0073c2-b9d2-41f3-883e-129a9b709966 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 03:01:51.568908 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:51.568908 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:51.568908 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:51.568908 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:51.568908 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:51.568908 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:51.568908 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:51.568908 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:51.568908 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:51.568908 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:51.568908 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:51.568908 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 25 03:01:51.568908 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 25 03:01:51.568908 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 25 03:01:51.581467 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 25 03:01:51.581467 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:51.581467 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:51.581467 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:51.581467 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:51.581467 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:51.581467 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:51.581467 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:51.581467 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:51.581467 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:51.581467 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:51.581467 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:51.581467 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:51.581467 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:51.581467 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:51.581467 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:51.581467 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:51.582689 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 25 03:01:51.582689 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 25 03:01:51.582689 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 25 03:01:51.582689 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 25 03:01:51.582689 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 25 03:01:51.582689 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 25 03:01:51.582689 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 03:01:51.582689 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:51.582689 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 498/979] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Aug 25 03:01:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 03:01:51.582689 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0073c2-b9d2-41f3-883e-129a9b709966 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:51.598147 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:51.653472 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2fb5b9-2e9e-4c04-96d3-c38c8fd8f69e None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:51.683313 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0073c2-b9d2-41f3-883e-129a9b709966 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:51.689474 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dcbbae-53d4-469d-80cf-20706d39f40c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:51.690105 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dcbbae-53d4-469d-80cf-20706d39f40c None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:51.690415 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dcbbae-53d4-469d-80cf-20706d39f40c None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:51.690690 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dcbbae-53d4-469d-80cf-20706d39f40c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:51.691279 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dcbbae-53d4-469d-80cf-20706d39f40c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:51.692644 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dcbbae-53d4-469d-80cf-20706d39f40c None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:51.701085 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-c4dcbbae-53d4-469d-80cf-20706d39f40c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 25 03:01:51.701085 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:51.701085 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 25 03:01:51.701085 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 25 03:01:51.701085 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 25 03:01:51.701085 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 25 03:01:51.701085 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 25 03:01:51.701085 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 03:01:51.701085 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 25 03:01:51.701085 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:51.701085 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 25 03:01:51.701085 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:51.701085 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:51.701085 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:51.701085 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:51.701085 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:51.701085 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:51.702300 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:51.702300 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:51.702300 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:51.702300 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:51.702300 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:51.702300 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:51.702300 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 25 03:01:51.702300 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 25 03:01:51.702300 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 25 03:01:51.702300 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 25 03:01:51.702300 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:51.702300 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:51.702300 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:51.702300 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:51.702300 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:51.702300 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:51.703408 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:51.703408 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:51.703408 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:51.703408 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:51.703408 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:51.703408 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:51.703408 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:51.703408 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:51.703408 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:51.703408 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:51.703408 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 25 03:01:51.703408 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 03:01:51.703408 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 25 03:01:51.703408 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:51.703408 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 482/980] 10.222.0.58 () {56 vars in 1126 bytes} [Thu Aug 25 03:01:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 03:01:51.704201 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba18c0f5-3e0d-442e-8343-22811c087135 None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:51.704201 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba18c0f5-3e0d-442e-8343-22811c087135 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:51.704517 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba18c0f5-3e0d-442e-8343-22811c087135 None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:51.705705 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 483/981] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:01:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:01:51.708582 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba18c0f5-3e0d-442e-8343-22811c087135 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:51.720936 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca1360c-8d8c-451e-a8bb-428a6f2a3649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:51.721263 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca1360c-8d8c-451e-a8bb-428a6f2a3649 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:51.721637 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca1360c-8d8c-451e-a8bb-428a6f2a3649 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:51.721806 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca1360c-8d8c-451e-a8bb-428a6f2a3649 None admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:51.722555 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca1360c-8d8c-451e-a8bb-428a6f2a3649 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:51.722857 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca1360c-8d8c-451e-a8bb-428a6f2a3649 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:51.724265 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca1360c-8d8c-451e-a8bb-428a6f2a3649 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:51.736613 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 499/982] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Aug 25 03:01:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:01:51.737814 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac9f7f0-c218-495f-8c0b-147a68bf9f24 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:51.738097 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac9f7f0-c218-495f-8c0b-147a68bf9f24 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:51.738463 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac9f7f0-c218-495f-8c0b-147a68bf9f24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:51.755839 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-9ac9f7f0-c218-495f-8c0b-147a68bf9f24 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:01:51.757021 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-9ac9f7f0-c218-495f-8c0b-147a68bf9f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:01:51.786552 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:51.809758 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba18c0f5-3e0d-442e-8343-22811c087135 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:51.823262 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 500/983] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:01:51 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 03:01:51.824095 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac9f7f0-c218-495f-8c0b-147a68bf9f24 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:51.832392 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:51.846132 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a47cee-7fbf-4e47-89bb-71ba73e61f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:51.846793 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a47cee-7fbf-4e47-89bb-71ba73e61f26 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:51.847075 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a47cee-7fbf-4e47-89bb-71ba73e61f26 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:51.847350 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a47cee-7fbf-4e47-89bb-71ba73e61f26 None admin] PATH_INFO: `/v3/services` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:51.847984 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a47cee-7fbf-4e47-89bb-71ba73e61f26 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:51.848289 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a47cee-7fbf-4e47-89bb-71ba73e61f26 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:51.849714 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a47cee-7fbf-4e47-89bb-71ba73e61f26 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:51.859384 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 484/984] 10.222.0.58 () {56 vars in 1122 bytes} [Thu Aug 25 03:01:51 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:51.860574 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a47cee-7fbf-4e47-89bb-71ba73e61f26 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:51.929719 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac9f7f0-c218-495f-8c0b-147a68bf9f24 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:51.963481 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d046034-0025-4aa8-a04d-704c46f375f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:51.964087 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d046034-0025-4aa8-a04d-704c46f375f9 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:51.964363 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d046034-0025-4aa8-a04d-704c46f375f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:51.964629 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d046034-0025-4aa8-a04d-704c46f375f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:51.965346 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d046034-0025-4aa8-a04d-704c46f375f9 None admin] RBAC: Adding query filter params (role.id=b8d76e033b4b43ccbe4e580f17be8cc9, scope.project.id=9fd4be43faec4e4191e0e4ec4a4588c3, user.id=712c35b765b64919aa3eaaf02892590d) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:51.965646 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d046034-0025-4aa8-a04d-704c46f375f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:51.967068 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d046034-0025-4aa8-a04d-704c46f375f9 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:51.970038 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a47cee-7fbf-4e47-89bb-71ba73e61f26 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:51.973199 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 501/985] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Aug 25 03:01:51 2022] GET /identity/v3/role_assignments?user.id=712c35b765b64919aa3eaaf02892590d&scope.project.id=9fd4be43faec4e4191e0e4ec4a4588c3&role.id=b8d76e033b4b43ccbe4e580f17be8cc9 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:52.012857 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750d9fa7-8457-437f-b875-ab89803e007d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:52.013482 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750d9fa7-8457-437f-b875-ab89803e007d None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:52.013764 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750d9fa7-8457-437f-b875-ab89803e007d None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:52.014038 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750d9fa7-8457-437f-b875-ab89803e007d None admin] PATH_INFO: `/v3/services` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:52.014752 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750d9fa7-8457-437f-b875-ab89803e007d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:52.015057 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750d9fa7-8457-437f-b875-ab89803e007d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:52.016456 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750d9fa7-8457-437f-b875-ab89803e007d None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:52.022339 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 485/986] 10.222.0.58 () {56 vars in 1122 bytes} [Thu Aug 25 03:01:51 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:01:52.028245 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d046034-0025-4aa8-a04d-704c46f375f9 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:52.096370 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750d9fa7-8457-437f-b875-ab89803e007d None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:52.100231 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:52.104915 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:52.140185 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d046034-0025-4aa8-a04d-704c46f375f9 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:52.179593 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f158a15a-aeb9-42cf-ac65-a66619d1576d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:52.180254 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f158a15a-aeb9-42cf-ac65-a66619d1576d None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:52.180482 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f158a15a-aeb9-42cf-ac65-a66619d1576d None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:52.180750 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f158a15a-aeb9-42cf-ac65-a66619d1576d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:52.181459 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f158a15a-aeb9-42cf-ac65-a66619d1576d None admin] RBAC: Adding query filter params (interface=admin, service_id=d1ad6cff0208431996519da505864a8c, region_id=RegionOne) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:52.181881 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f158a15a-aeb9-42cf-ac65-a66619d1576d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:52.183450 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f158a15a-aeb9-42cf-ac65-a66619d1576d None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:52.189763 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 502/987] 10.222.0.58 () {56 vars in 1249 bytes} [Thu Aug 25 03:01:52 2022] GET /identity/v3/endpoints?service_id=d1ad6cff0208431996519da505864a8c&interface=admin®ion_id=RegionOne => generated 193 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:52.195188 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f158a15a-aeb9-42cf-ac65-a66619d1576d None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:52.209871 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750d9fa7-8457-437f-b875-ab89803e007d None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:52.245251 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925d6f94-1162-4243-ae9f-5e8d62e79aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:52.246009 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d6f94-1162-4243-ae9f-5e8d62e79aad None admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:52.246412 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d6f94-1162-4243-ae9f-5e8d62e79aad None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:52.246777 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d6f94-1162-4243-ae9f-5e8d62e79aad None admin] PATH_INFO: `/v3/services` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:52.247519 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925d6f94-1162-4243-ae9f-5e8d62e79aad None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:52.248972 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925d6f94-1162-4243-ae9f-5e8d62e79aad None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:52.300735 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f158a15a-aeb9-42cf-ac65-a66619d1576d None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:52.316539 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 486/988] 10.222.0.58 () {60 vars in 1141 bytes} [Thu Aug 25 03:01:52 2022] POST /identity/v3/services => generated 242 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:01:52.339410 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:52.346355 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444a9e5b-8ac5-4921-9277-2551fdce8ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:52.347119 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444a9e5b-8ac5-4921-9277-2551fdce8ef6 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:52.347455 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444a9e5b-8ac5-4921-9277-2551fdce8ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:52.347812 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444a9e5b-8ac5-4921-9277-2551fdce8ef6 None admin] PATH_INFO: `/v3/services` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:52.348499 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444a9e5b-8ac5-4921-9277-2551fdce8ef6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:52.350274 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444a9e5b-8ac5-4921-9277-2551fdce8ef6 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:52.366585 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 503/989] 10.222.0.58 () {56 vars in 1087 bytes} [Thu Aug 25 03:01:52 2022] GET /identity/v3/services => generated 1727 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 25 03:01:53.918137 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b3ecff-7f38-4455-b079-c41d3c48ef53 None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:53.918686 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b3ecff-7f38-4455-b079-c41d3c48ef53 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:53.918766 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b3ecff-7f38-4455-b079-c41d3c48ef53 None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:53.920224 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 487/990] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:01:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:01:53.925721 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aaed4a-63e6-460d-8467-c8c9273a82d9 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:53.926109 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aaed4a-63e6-460d-8467-c8c9273a82d9 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:53.926270 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aaed4a-63e6-460d-8467-c8c9273a82d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:53.944721 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-72aaed4a-63e6-460d-8467-c8c9273a82d9 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:01:53.945901 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-72aaed4a-63e6-460d-8467-c8c9273a82d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:01:54.015463 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 504/991] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:01:53 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 03:01:54.259888 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b914c88-bb1f-4ebb-84d4-36522f6453dd None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:54.260228 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b914c88-bb1f-4ebb-84d4-36522f6453dd None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:54.260403 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b914c88-bb1f-4ebb-84d4-36522f6453dd None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:54.261722 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 488/992] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:01:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:01:54.266860 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef382d7-e434-42ea-84dd-8e061ae79271 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:54.267152 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef382d7-e434-42ea-84dd-8e061ae79271 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:54.267430 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef382d7-e434-42ea-84dd-8e061ae79271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:54.286017 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-eef382d7-e434-42ea-84dd-8e061ae79271 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:01:54.286310 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b914c88-bb1f-4ebb-84d4-36522f6453dd None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:54.287223 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-eef382d7-e434-42ea-84dd-8e061ae79271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:01:54.356947 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:54.366755 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 505/993] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:01:54 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 03:01:54.368021 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef50a8c-ef14-4040-8fca-5a5f1652cc92 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:54.368306 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef50a8c-ef14-4040-8fca-5a5f1652cc92 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:54.368579 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef50a8c-ef14-4040-8fca-5a5f1652cc92 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:54.373885 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 506/994] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:01:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:01:54.375229 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee41ccd-6158-4c3f-9037-45f03ea07985 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:54.375515 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee41ccd-6158-4c3f-9037-45f03ea07985 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:54.375857 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee41ccd-6158-4c3f-9037-45f03ea07985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:54.394233 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-6ee41ccd-6158-4c3f-9037-45f03ea07985 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:01:54.395603 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-6ee41ccd-6158-4c3f-9037-45f03ea07985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:01:54.398795 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b914c88-bb1f-4ebb-84d4-36522f6453dd None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:54.435110 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85d9f15-04e8-46f5-bc08-15e40b15f576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:54.435984 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85d9f15-04e8-46f5-bc08-15e40b15f576 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:54.436420 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85d9f15-04e8-46f5-bc08-15e40b15f576 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:54.436854 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85d9f15-04e8-46f5-bc08-15e40b15f576 None admin] PATH_INFO: `/v3/services/image` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:54.437630 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85d9f15-04e8-46f5-bc08-15e40b15f576 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:54.439173 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85d9f15-04e8-46f5-bc08-15e40b15f576 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:54.446904 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:54.448095 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-a85d9f15-04e8-46f5-bc08-15e40b15f576 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 25 03:01:54.448095 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:54.448095 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 25 03:01:54.448095 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 25 03:01:54.448095 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 25 03:01:54.448095 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 25 03:01:54.448095 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 25 03:01:54.448095 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 03:01:54.448095 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 25 03:01:54.448095 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:54.448095 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 25 03:01:54.448095 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:54.448095 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:54.448095 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:54.448095 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:54.448095 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:54.448095 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:54.449074 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:54.449074 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:54.449074 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:54.449074 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:54.449074 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:54.449074 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:54.449074 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 25 03:01:54.449074 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 25 03:01:54.449074 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 25 03:01:54.449074 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 25 03:01:54.449074 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:54.449074 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:54.449074 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:54.449074 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:54.449074 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:54.449074 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:54.449915 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:54.449915 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:54.449915 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:54.449915 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:54.449915 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:54.449915 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:54.449915 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:54.449915 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:54.449915 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:54.449915 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:54.449915 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 25 03:01:54.449915 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 03:01:54.449915 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 25 03:01:54.449915 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:54.449915 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 489/995] 10.222.0.58 () {56 vars in 1105 bytes} [Thu Aug 25 03:01:54 2022] GET /identity/v3/services/image => generated 86 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 03:01:54.455119 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85d9f15-04e8-46f5-bc08-15e40b15f576 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:54.468696 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 507/996] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:01:54 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 03:01:54.555382 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85d9f15-04e8-46f5-bc08-15e40b15f576 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:54.586382 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:54.594840 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a0b55c-93ea-4676-802b-48270d342f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:54.595448 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a0b55c-93ea-4676-802b-48270d342f86 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:54.595729 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a0b55c-93ea-4676-802b-48270d342f86 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:54.596002 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a0b55c-93ea-4676-802b-48270d342f86 None admin] PATH_INFO: `/v3/services` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:54.596631 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a0b55c-93ea-4676-802b-48270d342f86 None admin] RBAC: Adding query filter params (name=image) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:54.596932 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a0b55c-93ea-4676-802b-48270d342f86 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:54.598308 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a0b55c-93ea-4676-802b-48270d342f86 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:54.607809 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 490/997] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:01:54 2022] GET /identity/v3/services?name=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:01:54.609179 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee41ccd-6158-4c3f-9037-45f03ea07985 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:54.638049 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a0b55c-93ea-4676-802b-48270d342f86 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:54.728826 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee41ccd-6158-4c3f-9037-45f03ea07985 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:54.748651 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a0b55c-93ea-4676-802b-48270d342f86 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:54.766186 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe6a59d-f18f-4c20-bc7e-2c8f9646e2b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:54.766822 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe6a59d-f18f-4c20-bc7e-2c8f9646e2b8 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:54.767104 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe6a59d-f18f-4c20-bc7e-2c8f9646e2b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:54.767379 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe6a59d-f18f-4c20-bc7e-2c8f9646e2b8 None admin] PATH_INFO: `/v3/services` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:54.768003 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe6a59d-f18f-4c20-bc7e-2c8f9646e2b8 None admin] RBAC: Adding query filter params (type=image) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:54.768306 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe6a59d-f18f-4c20-bc7e-2c8f9646e2b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:54.769675 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe6a59d-f18f-4c20-bc7e-2c8f9646e2b8 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:54.773938 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 508/998] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:01:54 2022] GET /identity/v3/services?type=image => generated 122 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:01:54.774739 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe6a59d-f18f-4c20-bc7e-2c8f9646e2b8 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:54.782934 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778bfee1-695e-47b2-aa0d-acc91c683147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:54.783588 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778bfee1-695e-47b2-aa0d-acc91c683147 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:54.783878 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778bfee1-695e-47b2-aa0d-acc91c683147 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:54.784148 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778bfee1-695e-47b2-aa0d-acc91c683147 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:54.784749 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778bfee1-695e-47b2-aa0d-acc91c683147 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:54.786188 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778bfee1-695e-47b2-aa0d-acc91c683147 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:54.795814 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-778bfee1-695e-47b2-aa0d-acc91c683147 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 25 03:01:54.795814 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:54.795814 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 25 03:01:54.795814 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 25 03:01:54.795814 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 25 03:01:54.795814 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 25 03:01:54.795814 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 25 03:01:54.795814 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 03:01:54.795814 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 25 03:01:54.795814 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:54.795814 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 25 03:01:54.795814 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:54.795814 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:54.795814 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:54.795814 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:54.795814 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:54.795814 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:54.796946 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:54.796946 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:54.796946 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:54.796946 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:54.796946 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:54.796946 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:54.796946 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 25 03:01:54.796946 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 25 03:01:54.796946 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 25 03:01:54.796946 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 25 03:01:54.796946 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:54.796946 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:54.796946 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:54.796946 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:54.796946 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:54.796946 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:54.797794 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:54.797794 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:54.797794 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:54.797794 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:54.797794 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:54.797794 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:54.797794 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:54.797794 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:54.797794 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:54.797794 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:54.797794 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 25 03:01:54.797794 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 03:01:54.797794 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 25 03:01:54.797794 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:54.797794 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 491/999] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Aug 25 03:01:54 2022] GET /identity/v3/services/compute => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 03:01:54.803315 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778bfee1-695e-47b2-aa0d-acc91c683147 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:54.833863 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:54.869340 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:54.901233 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe6a59d-f18f-4c20-bc7e-2c8f9646e2b8 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:54.919106 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778bfee1-695e-47b2-aa0d-acc91c683147 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:54.942788 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2531b7-8e8a-4d7e-80cd-2b5e19e4b71c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:54.943980 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2531b7-8e8a-4d7e-80cd-2b5e19e4b71c None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:54.944257 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2531b7-8e8a-4d7e-80cd-2b5e19e4b71c None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:54.944528 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2531b7-8e8a-4d7e-80cd-2b5e19e4b71c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:54.945112 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2531b7-8e8a-4d7e-80cd-2b5e19e4b71c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:54.946545 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2531b7-8e8a-4d7e-80cd-2b5e19e4b71c None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:54.956195 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-9c2531b7-8e8a-4d7e-80cd-2b5e19e4b71c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 25 03:01:54.956195 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:54.956195 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 25 03:01:54.956195 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 25 03:01:54.956195 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 25 03:01:54.956195 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 25 03:01:54.956195 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 25 03:01:54.956195 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 03:01:54.956195 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 25 03:01:54.956195 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:54.956195 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 25 03:01:54.956195 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:54.956195 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:54.956195 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:54.956195 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:54.956195 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:54.956195 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:54.957374 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:54.957374 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:54.957374 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:54.957374 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:54.957374 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:54.957374 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:54.957374 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 25 03:01:54.957374 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 25 03:01:54.957374 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 25 03:01:54.957374 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 25 03:01:54.957374 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:54.957374 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:54.957374 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:54.957374 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:54.957374 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:54.957374 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:54.958241 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:54.958241 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:54.958241 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:54.958241 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:54.958241 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:54.958241 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:54.958241 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:54.958241 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:54.958241 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:54.958241 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:54.958241 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 25 03:01:54.958241 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 03:01:54.958241 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 25 03:01:54.958241 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:01:54.958241 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 509/1000] 10.222.0.58 () {56 vars in 1126 bytes} [Thu Aug 25 03:01:54 2022] GET /identity/v3/services/object-store => generated 93 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:54.963003 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2531b7-8e8a-4d7e-80cd-2b5e19e4b71c None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:54.963526 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2ccdcb-9b05-46df-880f-67fbc45f83a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:54.964147 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2ccdcb-9b05-46df-880f-67fbc45f83a9 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:54.964429 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2ccdcb-9b05-46df-880f-67fbc45f83a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:54.964706 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2ccdcb-9b05-46df-880f-67fbc45f83a9 None admin] PATH_INFO: `/v3/services` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:54.965348 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2ccdcb-9b05-46df-880f-67fbc45f83a9 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:54.965651 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2ccdcb-9b05-46df-880f-67fbc45f83a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:54.967442 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2ccdcb-9b05-46df-880f-67fbc45f83a9 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:54.973919 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 492/1001] 10.222.0.58 () {56 vars in 1112 bytes} [Thu Aug 25 03:01:54 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:01:54.979200 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2ccdcb-9b05-46df-880f-67fbc45f83a9 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:55.082920 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2531b7-8e8a-4d7e-80cd-2b5e19e4b71c None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:55.090570 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2ccdcb-9b05-46df-880f-67fbc45f83a9 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:55.098252 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:55.111562 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:55.119123 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8930d525-08ce-40a3-94d8-664940ac1eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:55.119738 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8930d525-08ce-40a3-94d8-664940ac1eb0 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:55.120019 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8930d525-08ce-40a3-94d8-664940ac1eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:55.120294 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8930d525-08ce-40a3-94d8-664940ac1eb0 None admin] PATH_INFO: `/v3/services` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:55.120930 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8930d525-08ce-40a3-94d8-664940ac1eb0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:55.121234 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8930d525-08ce-40a3-94d8-664940ac1eb0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:55.122625 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8930d525-08ce-40a3-94d8-664940ac1eb0 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:55.128064 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 510/1002] 10.222.0.58 () {56 vars in 1122 bytes} [Thu Aug 25 03:01:54 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:55.129287 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d28888-be69-4648-9c36-96f9a272a2d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:55.129976 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d28888-be69-4648-9c36-96f9a272a2d2 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:55.130239 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d28888-be69-4648-9c36-96f9a272a2d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:55.130570 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d28888-be69-4648-9c36-96f9a272a2d2 None admin] PATH_INFO: `/v3/services` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:55.131204 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d28888-be69-4648-9c36-96f9a272a2d2 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:55.131510 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d28888-be69-4648-9c36-96f9a272a2d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:55.132856 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d28888-be69-4648-9c36-96f9a272a2d2 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:55.133841 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8930d525-08ce-40a3-94d8-664940ac1eb0 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:55.138015 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 493/1003] 10.222.0.58 () {56 vars in 1112 bytes} [Thu Aug 25 03:01:54 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:01:55.144975 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d28888-be69-4648-9c36-96f9a272a2d2 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:55.244550 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8930d525-08ce-40a3-94d8-664940ac1eb0 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:55.251299 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d28888-be69-4648-9c36-96f9a272a2d2 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:55.286226 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79763b2d-76de-484c-aa4b-71aaa89bc30e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:55.286975 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79763b2d-76de-484c-aa4b-71aaa89bc30e None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:55.287192 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69de42c3-8fd5-4890-b919-7830d4d286cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:55.287319 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79763b2d-76de-484c-aa4b-71aaa89bc30e None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:55.287517 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79763b2d-76de-484c-aa4b-71aaa89bc30e None admin] PATH_INFO: `/v3/services` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:55.287768 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69de42c3-8fd5-4890-b919-7830d4d286cb None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:55.288052 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69de42c3-8fd5-4890-b919-7830d4d286cb None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:55.288179 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79763b2d-76de-484c-aa4b-71aaa89bc30e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:55.288328 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69de42c3-8fd5-4890-b919-7830d4d286cb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:55.288503 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79763b2d-76de-484c-aa4b-71aaa89bc30e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:55.289023 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69de42c3-8fd5-4890-b919-7830d4d286cb None admin] RBAC: Adding query filter params (interface=public, service_id=013f06dfced0409f828f50ff53144760, region_id=RegionOne) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:55.289334 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69de42c3-8fd5-4890-b919-7830d4d286cb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:55.289886 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79763b2d-76de-484c-aa4b-71aaa89bc30e None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:55.290747 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69de42c3-8fd5-4890-b919-7830d4d286cb None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:55.295226 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 511/1004] 10.222.0.58 () {56 vars in 1122 bytes} [Thu Aug 25 03:01:55 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:01:55.300461 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 494/1005] 10.222.0.58 () {56 vars in 1251 bytes} [Thu Aug 25 03:01:55 2022] GET /identity/v3/endpoints?service_id=013f06dfced0409f828f50ff53144760&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:01:55.300693 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79763b2d-76de-484c-aa4b-71aaa89bc30e None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:55.301208 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69de42c3-8fd5-4890-b919-7830d4d286cb None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:55.345342 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:55.355507 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:55.408991 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69de42c3-8fd5-4890-b919-7830d4d286cb None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:55.409782 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79763b2d-76de-484c-aa4b-71aaa89bc30e None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:55.446109 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530c5659-f5c5-407e-b9bf-4310cf022769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:55.446746 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530c5659-f5c5-407e-b9bf-4310cf022769 None admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:55.447024 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530c5659-f5c5-407e-b9bf-4310cf022769 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:55.447288 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530c5659-f5c5-407e-b9bf-4310cf022769 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:55.447941 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530c5659-f5c5-407e-b9bf-4310cf022769 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:55.449037 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb3372b-0397-4f7e-a479-cf3ae965d0e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:55.449295 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530c5659-f5c5-407e-b9bf-4310cf022769 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:55.449657 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb3372b-0397-4f7e-a479-cf3ae965d0e1 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:55.449996 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb3372b-0397-4f7e-a479-cf3ae965d0e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:55.450305 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb3372b-0397-4f7e-a479-cf3ae965d0e1 None admin] PATH_INFO: `/v3/services` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:55.450909 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb3372b-0397-4f7e-a479-cf3ae965d0e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:55.452297 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb3372b-0397-4f7e-a479-cf3ae965d0e1 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:55.459605 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 495/1006] 10.222.0.58 () {56 vars in 1087 bytes} [Thu Aug 25 03:01:55 2022] GET /identity/v3/services => generated 1727 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:01:55.526327 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 512/1007] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:01:55 2022] POST /identity/v3/endpoints => generated 325 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:01:56.688347 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf41da27-cc74-44d0-af91-3c6642144ae7 None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:56.688841 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf41da27-cc74-44d0-af91-3c6642144ae7 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:56.688920 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf41da27-cc74-44d0-af91-3c6642144ae7 None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:56.690393 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 496/1008] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:01:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:01:56.695302 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35fe7f1-96fe-46f2-acb4-092b51ed0243 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:56.695677 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35fe7f1-96fe-46f2-acb4-092b51ed0243 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:56.695859 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35fe7f1-96fe-46f2-acb4-092b51ed0243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:56.714404 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-c35fe7f1-96fe-46f2-acb4-092b51ed0243 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:01:56.715580 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-c35fe7f1-96fe-46f2-acb4-092b51ed0243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:01:56.787645 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 513/1009] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:01:56 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 03:01:57.054273 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf41da27-cc74-44d0-af91-3c6642144ae7 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:57.158774 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf41da27-cc74-44d0-af91-3c6642144ae7 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:57.171836 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:57.192133 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fe8833-d4ca-4c13-9bdc-e6a5e659723e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:57.192741 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fe8833-d4ca-4c13-9bdc-e6a5e659723e None admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:57.193022 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fe8833-d4ca-4c13-9bdc-e6a5e659723e None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:57.193297 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fe8833-d4ca-4c13-9bdc-e6a5e659723e None admin] PATH_INFO: `/v3/services` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:57.193956 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fe8833-d4ca-4c13-9bdc-e6a5e659723e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:57.195345 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fe8833-d4ca-4c13-9bdc-e6a5e659723e None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:57.287209 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 497/1010] 10.222.0.58 () {60 vars in 1141 bytes} [Thu Aug 25 03:01:57 2022] POST /identity/v3/services => generated 242 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:01:57.393432 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e39d9e-b246-4776-a218-8e46fced352c None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:57.393707 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e39d9e-b246-4776-a218-8e46fced352c None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:57.393946 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e39d9e-b246-4776-a218-8e46fced352c None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:57.395501 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 514/1011] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:01:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:01:57.401119 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0212d2c-41d3-4bbf-b6a4-b786855aeb95 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:57.401407 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0212d2c-41d3-4bbf-b6a4-b786855aeb95 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:57.401691 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0212d2c-41d3-4bbf-b6a4-b786855aeb95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:57.420616 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-c0212d2c-41d3-4bbf-b6a4-b786855aeb95 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:01:57.421834 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-c0212d2c-41d3-4bbf-b6a4-b786855aeb95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:01:57.457635 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cbe122-51a1-4104-a048-0ee411fb15d7 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:57.457919 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cbe122-51a1-4104-a048-0ee411fb15d7 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:57.458191 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cbe122-51a1-4104-a048-0ee411fb15d7 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:57.459564 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 515/1012] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:01:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 25 03:01:57.464765 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c4dd92-79df-433c-998b-9c391dd95fa9 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:57.465052 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c4dd92-79df-433c-998b-9c391dd95fa9 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:57.465330 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c4dd92-79df-433c-998b-9c391dd95fa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:57.485081 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-67c4dd92-79df-433c-998b-9c391dd95fa9 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:01:57.486264 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-67c4dd92-79df-433c-998b-9c391dd95fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:01:57.503078 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 498/1013] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:01:57 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 03:01:57.510702 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:57.556597 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 516/1014] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:01:57 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 25 03:01:57.779904 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0212d2c-41d3-4bbf-b6a4-b786855aeb95 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:57.832899 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c4dd92-79df-433c-998b-9c391dd95fa9 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:57.879297 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:57.883005 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0212d2c-41d3-4bbf-b6a4-b786855aeb95 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:57.918036 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97194dcd-0d48-4f2a-a4a8-b3c0b304c2df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:57.918703 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97194dcd-0d48-4f2a-a4a8-b3c0b304c2df None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:57.918986 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97194dcd-0d48-4f2a-a4a8-b3c0b304c2df None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:57.919246 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97194dcd-0d48-4f2a-a4a8-b3c0b304c2df None admin] PATH_INFO: `/v3/services/compute` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:57.919825 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97194dcd-0d48-4f2a-a4a8-b3c0b304c2df None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:57.921725 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97194dcd-0d48-4f2a-a4a8-b3c0b304c2df None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:57.930434 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-97194dcd-0d48-4f2a-a4a8-b3c0b304c2df None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 25 03:01:57.930434 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:57.930434 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 25 03:01:57.930434 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 25 03:01:57.930434 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 25 03:01:57.930434 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 25 03:01:57.930434 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 25 03:01:57.930434 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 03:01:57.930434 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 25 03:01:57.930434 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:57.930434 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 25 03:01:57.930434 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:57.930434 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:57.930434 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:57.930434 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:57.930434 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:57.930434 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:57.931434 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:57.931434 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:57.931434 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:57.931434 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:57.931434 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:57.931434 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:57.931434 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 25 03:01:57.931434 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 25 03:01:57.931434 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 25 03:01:57.931434 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 25 03:01:57.931434 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:57.931434 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:57.931434 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:57.931434 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:57.931434 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:57.931434 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:57.932280 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:57.932280 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:57.932280 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:57.932280 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:57.932280 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:57.932280 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:57.932280 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:57.932280 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:57.932280 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:57.932280 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:57.932280 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 25 03:01:57.932280 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 03:01:57.932280 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 25 03:01:57.932280 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:57.932280 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 499/1015] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Aug 25 03:01:57 2022] GET /identity/v3/services/compute => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:57.937797 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97194dcd-0d48-4f2a-a4a8-b3c0b304c2df None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:57.941099 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c4dd92-79df-433c-998b-9c391dd95fa9 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:57.975788 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc39dc3-2b76-448e-9fd9-2ae21a9bc706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:57.979508 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc39dc3-2b76-448e-9fd9-2ae21a9bc706 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:57.979508 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc39dc3-2b76-448e-9fd9-2ae21a9bc706 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:57.979508 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc39dc3-2b76-448e-9fd9-2ae21a9bc706 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:57.979508 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc39dc3-2b76-448e-9fd9-2ae21a9bc706 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:57.979508 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc39dc3-2b76-448e-9fd9-2ae21a9bc706 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:57.981305 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 517/1016] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:01:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:01:57.987067 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc39dc3-2b76-448e-9fd9-2ae21a9bc706 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:58.016146 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:58.044089 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97194dcd-0d48-4f2a-a4a8-b3c0b304c2df None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:58.078967 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12231ace-59dd-4a69-a1b0-48f7b7a6aac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:58.079571 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12231ace-59dd-4a69-a1b0-48f7b7a6aac1 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:58.079846 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12231ace-59dd-4a69-a1b0-48f7b7a6aac1 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:58.080115 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12231ace-59dd-4a69-a1b0-48f7b7a6aac1 None admin] PATH_INFO: `/v3/services` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:58.080739 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12231ace-59dd-4a69-a1b0-48f7b7a6aac1 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:58.081037 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12231ace-59dd-4a69-a1b0-48f7b7a6aac1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:58.082482 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12231ace-59dd-4a69-a1b0-48f7b7a6aac1 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:58.087329 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 500/1017] 10.222.0.58 () {56 vars in 1112 bytes} [Thu Aug 25 03:01:57 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:01:58.092755 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12231ace-59dd-4a69-a1b0-48f7b7a6aac1 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:58.096468 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc39dc3-2b76-448e-9fd9-2ae21a9bc706 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:58.121942 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:58.132165 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a841d4db-e28e-4a25-a99d-5149b25bcfcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:58.132780 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a841d4db-e28e-4a25-a99d-5149b25bcfcc None admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:58.133060 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a841d4db-e28e-4a25-a99d-5149b25bcfcc None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:58.133336 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a841d4db-e28e-4a25-a99d-5149b25bcfcc None admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:58.134468 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a841d4db-e28e-4a25-a99d-5149b25bcfcc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:58.135583 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a841d4db-e28e-4a25-a99d-5149b25bcfcc None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:58.194677 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12231ace-59dd-4a69-a1b0-48f7b7a6aac1 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:58.228415 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3684c6d9-6d4e-4e81-b101-8c9c2fdd335b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:58.229021 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3684c6d9-6d4e-4e81-b101-8c9c2fdd335b None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:58.229303 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3684c6d9-6d4e-4e81-b101-8c9c2fdd335b None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:58.229618 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3684c6d9-6d4e-4e81-b101-8c9c2fdd335b None admin] PATH_INFO: `/v3/services` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:58.230204 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3684c6d9-6d4e-4e81-b101-8c9c2fdd335b None admin] RBAC: Adding query filter params (type=compute) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:58.230535 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3684c6d9-6d4e-4e81-b101-8c9c2fdd335b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:58.242549 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 518/1018] 10.222.0.58 () {60 vars in 1141 bytes} [Thu Aug 25 03:01:57 2022] POST /identity/v3/projects => generated 311 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:01:58.242771 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3684c6d9-6d4e-4e81-b101-8c9c2fdd335b None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:58.242771 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 501/1019] 10.222.0.58 () {56 vars in 1112 bytes} [Thu Aug 25 03:01:58 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:01:58.243040 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a841d4db-e28e-4a25-a99d-5149b25bcfcc None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:58.332281 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:58.356457 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a841d4db-e28e-4a25-a99d-5149b25bcfcc None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:58.394226 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f945c5fe-c55f-42ae-a109-5497653ac413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:58.395074 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f945c5fe-c55f-42ae-a109-5497653ac413 None admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:58.395772 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f945c5fe-c55f-42ae-a109-5497653ac413 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:58.396052 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f945c5fe-c55f-42ae-a109-5497653ac413 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:58.396844 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f945c5fe-c55f-42ae-a109-5497653ac413 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:58.398117 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f945c5fe-c55f-42ae-a109-5497653ac413 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:58.474717 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 519/1020] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:01:58 2022] POST /identity/v3/endpoints => generated 334 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:01:59.246854 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4613b585-52c2-4301-91e2-dc2c735f1a57 None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:59.247364 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4613b585-52c2-4301-91e2-dc2c735f1a57 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:59.247472 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4613b585-52c2-4301-91e2-dc2c735f1a57 None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:59.248873 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 502/1021] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:01:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:01:59.254101 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b851f0c-5d6f-4892-beb7-d55623330d29 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:59.254545 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b851f0c-5d6f-4892-beb7-d55623330d29 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:59.254822 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b851f0c-5d6f-4892-beb7-d55623330d29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:59.274043 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-3b851f0c-5d6f-4892-beb7-d55623330d29 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:01:59.275271 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-3b851f0c-5d6f-4892-beb7-d55623330d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:01:59.349426 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 520/1022] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:01:59 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 03:01:59.614856 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4613b585-52c2-4301-91e2-dc2c735f1a57 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:59.712796 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:59.723201 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4613b585-52c2-4301-91e2-dc2c735f1a57 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:59.757415 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28079e5a-065b-4c94-bb08-da704ab3b769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:59.758105 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28079e5a-065b-4c94-bb08-da704ab3b769 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:59.758455 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28079e5a-065b-4c94-bb08-da704ab3b769 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:59.758767 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28079e5a-065b-4c94-bb08-da704ab3b769 None admin] PATH_INFO: `/v3/services/image` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:59.759373 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28079e5a-065b-4c94-bb08-da704ab3b769 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:59.760742 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28079e5a-065b-4c94-bb08-da704ab3b769 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:59.770161 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-28079e5a-065b-4c94-bb08-da704ab3b769 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 25 03:01:59.770161 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:59.770161 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 25 03:01:59.770161 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 25 03:01:59.770161 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 25 03:01:59.770161 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 25 03:01:59.770161 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 25 03:01:59.770161 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 03:01:59.770161 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 25 03:01:59.770161 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:59.770161 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 25 03:01:59.770161 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:59.770161 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:01:59.770161 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:01:59.770161 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:01:59.770161 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:01:59.770161 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:01:59.771393 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:01:59.771393 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:01:59.771393 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:01:59.771393 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:01:59.771393 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:01:59.771393 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:01:59.771393 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 25 03:01:59.771393 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 25 03:01:59.771393 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 25 03:01:59.771393 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 25 03:01:59.771393 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:01:59.771393 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:01:59.771393 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:01:59.771393 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:01:59.771393 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:01:59.771393 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:01:59.772466 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:01:59.772466 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:01:59.772466 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:01:59.772466 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:01:59.772466 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:01:59.772466 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:01:59.772466 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:01:59.772466 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:01:59.772466 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:01:59.772466 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:01:59.772466 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 25 03:01:59.772466 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 03:01:59.772466 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 25 03:01:59.772466 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:01:59.779513 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 503/1023] 10.222.0.58 () {56 vars in 1105 bytes} [Thu Aug 25 03:01:59 2022] GET /identity/v3/services/image => generated 86 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:01:59.781092 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b851f0c-5d6f-4892-beb7-d55623330d29 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:59.839753 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:59.883237 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b851f0c-5d6f-4892-beb7-d55623330d29 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:59.917237 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e88f0e-4884-4e04-b653-0544c4579895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:59.917850 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e88f0e-4884-4e04-b653-0544c4579895 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:59.918185 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e88f0e-4884-4e04-b653-0544c4579895 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:59.918530 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e88f0e-4884-4e04-b653-0544c4579895 None admin] PATH_INFO: `/v3/services` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:59.919162 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e88f0e-4884-4e04-b653-0544c4579895 None admin] RBAC: Adding query filter params (name=image) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:01:59.919467 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e88f0e-4884-4e04-b653-0544c4579895 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:59.920833 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e88f0e-4884-4e04-b653-0544c4579895 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:59.930865 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 521/1024] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:01:59 2022] GET /identity/v3/services?name=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:01:59.932380 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28079e5a-065b-4c94-bb08-da704ab3b769 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:00.035143 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28079e5a-065b-4c94-bb08-da704ab3b769 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:00.069293 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323c8f4a-074b-4193-9ac5-c40e95023073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:00.069903 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323c8f4a-074b-4193-9ac5-c40e95023073 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:00.070181 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323c8f4a-074b-4193-9ac5-c40e95023073 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:00.070475 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323c8f4a-074b-4193-9ac5-c40e95023073 None admin] PATH_INFO: `/v3/services` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:00.071110 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323c8f4a-074b-4193-9ac5-c40e95023073 None admin] RBAC: Adding query filter params (type=image) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:02:00.071412 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323c8f4a-074b-4193-9ac5-c40e95023073 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:00.072778 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323c8f4a-074b-4193-9ac5-c40e95023073 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:00.082871 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 504/1025] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:01:59 2022] GET /identity/v3/services?type=image => generated 350 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:02:00.084136 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e88f0e-4884-4e04-b653-0544c4579895 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:00.185749 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e88f0e-4884-4e04-b653-0544c4579895 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:00.193097 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656b9f54-99e3-4dfb-8ed6-6cf52aa4eb45 None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:00.193385 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656b9f54-99e3-4dfb-8ed6-6cf52aa4eb45 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:00.193662 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656b9f54-99e3-4dfb-8ed6-6cf52aa4eb45 None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:00.195160 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 505/1026] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:02:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:02:00.199888 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e78153a-11aa-43db-a64e-9b6e2b6c45e3 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:00.200170 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e78153a-11aa-43db-a64e-9b6e2b6c45e3 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:00.200444 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e78153a-11aa-43db-a64e-9b6e2b6c45e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:00.210730 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:00.218458 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-9e78153a-11aa-43db-a64e-9b6e2b6c45e3 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:02:00.224052 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-9e78153a-11aa-43db-a64e-9b6e2b6c45e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:02:00.224211 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84141f5-6152-44aa-961b-1443ce141890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:00.224211 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84141f5-6152-44aa-961b-1443ce141890 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:00.224569 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84141f5-6152-44aa-961b-1443ce141890 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:00.224865 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84141f5-6152-44aa-961b-1443ce141890 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:00.225567 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84141f5-6152-44aa-961b-1443ce141890 None admin] RBAC: Adding query filter params (interface=public, service_id=608fe511797c40eb84cffa00a2e30d81, region_id=RegionOne) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:02:00.225879 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84141f5-6152-44aa-961b-1443ce141890 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:00.227384 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84141f5-6152-44aa-961b-1443ce141890 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:00.237410 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 522/1027] 10.222.0.58 () {56 vars in 1251 bytes} [Thu Aug 25 03:02:00 2022] GET /identity/v3/endpoints?service_id=608fe511797c40eb84cffa00a2e30d81&interface=public®ion_id=RegionOne => generated 194 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:02:00.237576 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:00.239805 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84141f5-6152-44aa-961b-1443ce141890 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:00.294387 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 506/1028] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:02:00 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 25 03:02:00.341313 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84141f5-6152-44aa-961b-1443ce141890 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:00.375355 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6f342d-1715-469d-91aa-fafb43802dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:00.375973 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6f342d-1715-469d-91aa-fafb43802dc2 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:00.376251 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6f342d-1715-469d-91aa-fafb43802dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:00.376528 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6f342d-1715-469d-91aa-fafb43802dc2 None admin] PATH_INFO: `/v3/services` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:00.377126 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6f342d-1715-469d-91aa-fafb43802dc2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:00.378514 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6f342d-1715-469d-91aa-fafb43802dc2 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:00.390521 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 523/1029] 10.222.0.58 () {56 vars in 1087 bytes} [Thu Aug 25 03:02:00 2022] GET /identity/v3/services => generated 1957 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 25 03:02:00.461356 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62bd119-1fc5-4266-9f83-4aa1a31d8bf6 None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:00.461547 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62bd119-1fc5-4266-9f83-4aa1a31d8bf6 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:00.461812 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62bd119-1fc5-4266-9f83-4aa1a31d8bf6 None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:00.463348 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 507/1030] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:02:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:02:00.469321 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c68b6c-a6ba-4fca-aa9d-09df1d82b845 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:00.469611 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c68b6c-a6ba-4fca-aa9d-09df1d82b845 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:00.469887 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c68b6c-a6ba-4fca-aa9d-09df1d82b845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:00.488961 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-b6c68b6c-a6ba-4fca-aa9d-09df1d82b845 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:02:00.490176 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-b6c68b6c-a6ba-4fca-aa9d-09df1d82b845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:02:00.541303 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:00.562384 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 524/1031] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:02:00 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 03:02:00.566101 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62bd119-1fc5-4266-9f83-4aa1a31d8bf6 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:00.672700 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62bd119-1fc5-4266-9f83-4aa1a31d8bf6 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:00.705784 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f83405-1113-40d4-bc45-bc67ff4c45fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:00.706447 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f83405-1113-40d4-bc45-bc67ff4c45fc None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:00.706728 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f83405-1113-40d4-bc45-bc67ff4c45fc None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:00.706999 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f83405-1113-40d4-bc45-bc67ff4c45fc None admin] PATH_INFO: `/v3/domains/default` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:00.708103 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f83405-1113-40d4-bc45-bc67ff4c45fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:00.709529 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f83405-1113-40d4-bc45-bc67ff4c45fc None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:00.711287 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 508/1032] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:02:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:02:00.716840 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c68b6c-a6ba-4fca-aa9d-09df1d82b845 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:00.815613 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c68b6c-a6ba-4fca-aa9d-09df1d82b845 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:00.844521 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f83405-1113-40d4-bc45-bc67ff4c45fc None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:00.852363 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38cac1f-f236-4739-87cf-503eb3edea7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:00.853467 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38cac1f-f236-4739-87cf-503eb3edea7b None admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:00.853750 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38cac1f-f236-4739-87cf-503eb3edea7b None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:00.854028 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38cac1f-f236-4739-87cf-503eb3edea7b None admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:00.854841 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38cac1f-f236-4739-87cf-503eb3edea7b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:00.856296 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38cac1f-f236-4739-87cf-503eb3edea7b None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:00.864301 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:00.938412 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 525/1033] 10.222.0.58 () {60 vars in 1132 bytes} [Thu Aug 25 03:02:00 2022] POST /identity/v3/users => generated 285 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:02:00.955122 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f83405-1113-40d4-bc45-bc67ff4c45fc None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:00.994954 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd528a0-3ff6-4536-ba3e-07c5500e3197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:00.995520 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd528a0-3ff6-4536-ba3e-07c5500e3197 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:00.995800 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd528a0-3ff6-4536-ba3e-07c5500e3197 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:00.996074 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd528a0-3ff6-4536-ba3e-07c5500e3197 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:01.008376 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd528a0-3ff6-4536-ba3e-07c5500e3197 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:02:01.013378 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-7fd528a0-3ff6-4536-ba3e-07c5500e3197 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 25 03:02:01.013378 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:02:01.013378 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:02:01.013378 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:02:01.013378 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:02:01.013378 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:02:01.013378 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:02:01.013378 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:02:01.013378 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:02:01.013378 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:02:01.013378 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:02:01.013378 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:02:01.013378 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:02:01.013378 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:02:01.013378 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:02:01.013378 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:02:01.014390 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:02:01.014390 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:02:01.014390 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:02:01.014390 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:02:01.014390 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:02:01.014390 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:02:01.014390 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:02:01.014390 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:02:01.014390 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:02:01.014390 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:02:01.014390 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:02:01.014390 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:02:01.014390 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:02:01.014390 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:02:01.014390 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:02:01.014390 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:02:01.019360 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:02:01.019360 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:02:01.019360 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:02:01.019360 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:02:01.019360 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:02:01.019360 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:02:01.019360 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:02:01.019360 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:02:01.019360 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:02:01.019360 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:02:01.019360 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 25 03:02:01.019360 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:02:01.019360 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 509/1034] 10.222.0.58 () {56 vars in 1120 bytes} [Thu Aug 25 03:02:00 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 03:02:01.021939 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38cac1f-f236-4739-87cf-503eb3edea7b None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:01.060968 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:01.131760 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38cac1f-f236-4739-87cf-503eb3edea7b None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:01.167957 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e9b39f-aed1-465d-9b60-83346fc2548d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:01.168567 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e9b39f-aed1-465d-9b60-83346fc2548d None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:01.168846 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e9b39f-aed1-465d-9b60-83346fc2548d None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:01.169123 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e9b39f-aed1-465d-9b60-83346fc2548d None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:01.169787 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e9b39f-aed1-465d-9b60-83346fc2548d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:02:01.170089 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e9b39f-aed1-465d-9b60-83346fc2548d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:01.171511 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e9b39f-aed1-465d-9b60-83346fc2548d None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:01.179747 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 526/1035] 10.222.0.58 () {56 vars in 1115 bytes} [Thu Aug 25 03:02:01 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:02:01.185555 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd528a0-3ff6-4536-ba3e-07c5500e3197 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:01.264510 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:01.289401 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd528a0-3ff6-4536-ba3e-07c5500e3197 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:01.331307 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f8106e-0bfa-480b-be3b-dfc29ffd4056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:01.331965 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f8106e-0bfa-480b-be3b-dfc29ffd4056 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:01.332194 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f8106e-0bfa-480b-be3b-dfc29ffd4056 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:01.332464 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f8106e-0bfa-480b-be3b-dfc29ffd4056 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:01.342449 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f8106e-0bfa-480b-be3b-dfc29ffd4056 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:01.344423 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f8106e-0bfa-480b-be3b-dfc29ffd4056 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:01.359388 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-b9f8106e-0bfa-480b-be3b-dfc29ffd4056 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:02:01.359388 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:02:01.359388 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:02:01.359388 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:02:01.359388 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:02:01.359388 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:02:01.359388 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:02:01.359388 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:02:01.359388 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:02:01.359388 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:02:01.359388 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:02:01.359388 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:02:01.359388 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 03:02:01.359388 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 03:02:01.359388 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 03:02:01.359388 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 03:02:01.360405 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:02:01.360405 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:02:01.360405 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:02:01.360405 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:02:01.360405 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:02:01.360405 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:02:01.360405 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:02:01.360405 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:02:01.360405 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:02:01.360405 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:02:01.360405 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:02:01.360405 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:02:01.360405 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:02:01.360405 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:02:01.360405 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:02:01.360405 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:02:01.360405 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 03:02:01.361396 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 03:02:01.361396 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:02:01.361396 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:02:01.361396 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 510/1036] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:02:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 03:02:01.368017 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e9b39f-aed1-465d-9b60-83346fc2548d None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:01.467793 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:01.472550 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e9b39f-aed1-465d-9b60-83346fc2548d None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:01.507914 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86137e79-7f42-4def-bb80-44380e258931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:01.508533 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86137e79-7f42-4def-bb80-44380e258931 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:01.508815 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86137e79-7f42-4def-bb80-44380e258931 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:01.509092 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86137e79-7f42-4def-bb80-44380e258931 None admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:01.509733 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86137e79-7f42-4def-bb80-44380e258931 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:02:01.510038 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86137e79-7f42-4def-bb80-44380e258931 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:01.511435 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86137e79-7f42-4def-bb80-44380e258931 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:01.528617 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 527/1037] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Aug 25 03:02:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:02:01.530269 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f8106e-0bfa-480b-be3b-dfc29ffd4056 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:01.658546 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f8106e-0bfa-480b-be3b-dfc29ffd4056 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:01.685761 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:01.693447 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bedb83-6ae9-425c-9fe3-e9750da5713b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:01.694076 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bedb83-6ae9-425c-9fe3-e9750da5713b None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:01.694385 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bedb83-6ae9-425c-9fe3-e9750da5713b None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:01.694664 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bedb83-6ae9-425c-9fe3-e9750da5713b None admin] PATH_INFO: `/v3/users/nova` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:01.701798 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bedb83-6ae9-425c-9fe3-e9750da5713b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:01.704080 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bedb83-6ae9-425c-9fe3-e9750da5713b None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:01.714641 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-b4bedb83-6ae9-425c-9fe3-e9750da5713b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 25 03:02:01.714641 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:02:01.714641 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:02:01.714641 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:02:01.714641 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:02:01.714641 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:02:01.714641 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:02:01.714641 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:02:01.714641 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:02:01.714641 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:02:01.714641 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:02:01.714641 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:02:01.714641 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 25 03:02:01.714641 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 25 03:02:01.714641 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 25 03:02:01.714641 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 25 03:02:01.715579 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:02:01.715579 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:02:01.715579 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 25 03:02:01.715579 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 03:02:01.715579 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 25 03:02:01.715579 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 03:02:01.715579 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:02:01.715579 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:02:01.715579 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:02:01.715579 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:02:01.715579 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:02:01.715579 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:02:01.715579 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:02:01.715579 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:02:01.715579 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:02:01.715579 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:02:01.715579 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:02:01.715579 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:02:01.716645 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:02:01.716645 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:02:01.716645 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 25 03:02:01.716645 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 25 03:02:01.716645 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 25 03:02:01.716645 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 25 03:02:01.716645 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 25 03:02:01.716645 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 25 03:02:01.716645 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 25 03:02:01.716645 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:02:01.718578 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 511/1038] 10.222.0.58 () {56 vars in 1093 bytes} [Thu Aug 25 03:02:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:02:01.723731 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86137e79-7f42-4def-bb80-44380e258931 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:01.826111 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86137e79-7f42-4def-bb80-44380e258931 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:01.859928 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f0f1e7-cef7-4689-8d9f-60caa6b4a309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:01.860536 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f0f1e7-cef7-4689-8d9f-60caa6b4a309 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:01.860815 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f0f1e7-cef7-4689-8d9f-60caa6b4a309 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:01.861087 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f0f1e7-cef7-4689-8d9f-60caa6b4a309 None admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:01.861815 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f0f1e7-cef7-4689-8d9f-60caa6b4a309 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:02:01.862118 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f0f1e7-cef7-4689-8d9f-60caa6b4a309 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:01.863565 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f0f1e7-cef7-4689-8d9f-60caa6b4a309 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:01.886315 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 528/1039] 10.222.0.58 () {56 vars in 1133 bytes} [Thu Aug 25 03:02:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:02:01.888182 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bedb83-6ae9-425c-9fe3-e9750da5713b None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:01.996050 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bedb83-6ae9-425c-9fe3-e9750da5713b None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:02.030512 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30187c10-a629-4efc-a45b-e08ae5cc85ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:02.031128 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30187c10-a629-4efc-a45b-e08ae5cc85ef None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:02.031417 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30187c10-a629-4efc-a45b-e08ae5cc85ef None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:02.031688 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30187c10-a629-4efc-a45b-e08ae5cc85ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:02.087504 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:02.101079 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30187c10-a629-4efc-a45b-e08ae5cc85ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:02.102650 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30187c10-a629-4efc-a45b-e08ae5cc85ef None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:02.207708 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 512/1040] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Aug 25 03:02:01 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 25 03:02:02.209378 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f0f1e7-cef7-4689-8d9f-60caa6b4a309 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:02.227837 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:02.309649 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f0f1e7-cef7-4689-8d9f-60caa6b4a309 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:02.324106 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed56719c-29d6-475e-ae86-2c4bf61f50a8 None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:02.324394 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed56719c-29d6-475e-ae86-2c4bf61f50a8 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:02.324666 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed56719c-29d6-475e-ae86-2c4bf61f50a8 None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:02.325981 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 513/1041] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:02:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 25 03:02:02.331531 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96efdba-21c4-4123-a7e8-3f60b9eb571e None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:02.332092 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96efdba-21c4-4123-a7e8-3f60b9eb571e None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:02.332092 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96efdba-21c4-4123-a7e8-3f60b9eb571e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:02.349121 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b060eee2-8b8b-43dc-9fc1-240c1b665c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:02.349683 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b060eee2-8b8b-43dc-9fc1-240c1b665c26 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:02.349965 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b060eee2-8b8b-43dc-9fc1-240c1b665c26 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:02.350240 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b060eee2-8b8b-43dc-9fc1-240c1b665c26 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:02.350457 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-e96efdba-21c4-4123-a7e8-3f60b9eb571e None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:02:02.351624 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-e96efdba-21c4-4123-a7e8-3f60b9eb571e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:02:02.361420 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b060eee2-8b8b-43dc-9fc1-240c1b665c26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:02.362895 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b060eee2-8b8b-43dc-9fc1-240c1b665c26 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:02.377638 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-b060eee2-8b8b-43dc-9fc1-240c1b665c26 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:02:02.377638 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:02:02.377638 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:02:02.377638 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:02:02.377638 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:02:02.377638 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:02:02.377638 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:02:02.377638 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:02:02.377638 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:02:02.377638 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:02:02.377638 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:02:02.377638 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:02:02.377638 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 03:02:02.377638 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 03:02:02.377638 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 03:02:02.377638 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 03:02:02.379278 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:02:02.379278 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:02:02.379278 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:02:02.379278 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:02:02.379278 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:02:02.379278 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:02:02.379278 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:02:02.379278 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:02:02.379278 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:02:02.379278 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:02:02.379278 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:02:02.379278 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:02:02.379278 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:02:02.379278 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:02:02.379278 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:02:02.379278 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:02:02.379278 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 03:02:02.380412 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 03:02:02.380412 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:02:02.380412 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:02:02.383310 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 529/1042] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:02:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:02:02.384538 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b060eee2-8b8b-43dc-9fc1-240c1b665c26 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:02.426496 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 514/1043] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:02:02 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 25 03:02:02.454661 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:02.486842 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b060eee2-8b8b-43dc-9fc1-240c1b665c26 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:02.522153 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a101fe-84d9-4e18-9dda-7c261fbb0af2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:02.522786 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a101fe-84d9-4e18-9dda-7c261fbb0af2 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:02.523058 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a101fe-84d9-4e18-9dda-7c261fbb0af2 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:02.523322 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a101fe-84d9-4e18-9dda-7c261fbb0af2 None admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:02.523952 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a101fe-84d9-4e18-9dda-7c261fbb0af2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:02:02.524249 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a101fe-84d9-4e18-9dda-7c261fbb0af2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:02.525593 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a101fe-84d9-4e18-9dda-7c261fbb0af2 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:02.542267 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 530/1044] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Aug 25 03:02:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:02:02.544033 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96efdba-21c4-4123-a7e8-3f60b9eb571e None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:02.563093 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:02.646946 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96efdba-21c4-4123-a7e8-3f60b9eb571e None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:02.681408 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec1dea7-630e-4c1e-b7f8-f42a4d65ba10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:02.682041 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec1dea7-630e-4c1e-b7f8-f42a4d65ba10 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:02.682340 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec1dea7-630e-4c1e-b7f8-f42a4d65ba10 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:02.682612 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec1dea7-630e-4c1e-b7f8-f42a4d65ba10 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:02.689336 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec1dea7-630e-4c1e-b7f8-f42a4d65ba10 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:02.690807 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec1dea7-630e-4c1e-b7f8-f42a4d65ba10 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:02.698420 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a101fe-84d9-4e18-9dda-7c261fbb0af2 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:02.701552 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-3ec1dea7-630e-4c1e-b7f8-f42a4d65ba10 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 03:02:02.701552 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:02:02.701552 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:02:02.701552 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:02:02.701552 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:02:02.701552 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:02:02.701552 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:02:02.701552 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:02:02.701552 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:02:02.701552 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:02:02.701552 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:02:02.701552 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:02:02.701552 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 25 03:02:02.701552 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 25 03:02:02.701552 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 25 03:02:02.702477 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 25 03:02:02.702477 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:02:02.702477 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:02:02.702477 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:02:02.702477 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:02:02.702477 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:02:02.702477 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:02:02.702477 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:02:02.702477 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:02:02.702477 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:02:02.702477 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:02:02.702477 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:02:02.702477 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:02:02.702477 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:02:02.702477 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:02:02.702477 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:02:02.702477 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:02:02.703818 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 25 03:02:02.703818 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 25 03:02:02.703818 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 25 03:02:02.703818 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 25 03:02:02.703818 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 25 03:02:02.703818 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 25 03:02:02.703818 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 03:02:02.703818 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:02:02.703818 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 515/1045] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Aug 25 03:02:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:02:02.708826 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec1dea7-630e-4c1e-b7f8-f42a4d65ba10 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:02.792511 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:02.806345 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a101fe-84d9-4e18-9dda-7c261fbb0af2 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:02.816298 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec1dea7-630e-4c1e-b7f8-f42a4d65ba10 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:02.833925 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:02.842062 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d5654f-e013-4063-97b1-79ae24e79669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:02.842787 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d5654f-e013-4063-97b1-79ae24e79669 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:02.843048 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d5654f-e013-4063-97b1-79ae24e79669 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:02.843358 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d5654f-e013-4063-97b1-79ae24e79669 None admin] PATH_INFO: `/v3/services/image` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:02.843946 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d5654f-e013-4063-97b1-79ae24e79669 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:02.848029 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d5654f-e013-4063-97b1-79ae24e79669 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:02.853818 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-d0d5654f-e013-4063-97b1-79ae24e79669 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 25 03:02:02.853818 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:02:02.853818 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 25 03:02:02.853818 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 25 03:02:02.853818 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 25 03:02:02.853818 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 25 03:02:02.853818 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 25 03:02:02.853818 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 03:02:02.853818 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 25 03:02:02.853818 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:02:02.853818 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 25 03:02:02.853818 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:02:02.853818 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:02:02.853818 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:02:02.853818 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:02:02.853818 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:02:02.853818 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:02:02.855059 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:02:02.855059 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:02:02.855059 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:02:02.855059 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:02:02.855059 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:02:02.855059 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:02:02.855059 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 25 03:02:02.855059 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 25 03:02:02.855059 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 25 03:02:02.855059 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 25 03:02:02.855059 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:02:02.855059 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:02:02.855059 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:02:02.855059 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:02:02.855059 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:02:02.855059 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:02:02.856191 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:02:02.856191 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:02:02.856191 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:02:02.856191 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:02:02.856191 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:02:02.856191 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:02:02.856191 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:02:02.856191 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:02:02.856191 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:02:02.856191 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:02:02.856191 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 25 03:02:02.856191 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 03:02:02.856191 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 25 03:02:02.856191 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:02:02.857497 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b63f9f5-662c-4a93-a2f7-11f27b247e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:02.858305 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b63f9f5-662c-4a93-a2f7-11f27b247e8d None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:02.859112 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b63f9f5-662c-4a93-a2f7-11f27b247e8d None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:02.859388 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b63f9f5-662c-4a93-a2f7-11f27b247e8d None admin] PATH_INFO: `/v3/projects` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:02.860545 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 531/1046] 10.222.0.58 () {56 vars in 1105 bytes} [Thu Aug 25 03:02:02 2022] GET /identity/v3/services/image => generated 86 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 03:02:02.860673 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b63f9f5-662c-4a93-a2f7-11f27b247e8d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:02:02.860854 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b63f9f5-662c-4a93-a2f7-11f27b247e8d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:02.861228 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d5654f-e013-4063-97b1-79ae24e79669 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:02.862413 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b63f9f5-662c-4a93-a2f7-11f27b247e8d None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:02.882112 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 516/1047] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Aug 25 03:02:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:02:02.883164 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b63f9f5-662c-4a93-a2f7-11f27b247e8d None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:02.973379 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d5654f-e013-4063-97b1-79ae24e79669 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:02.983240 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b63f9f5-662c-4a93-a2f7-11f27b247e8d None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:03.007372 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cf77a0-78b9-4322-9278-6be2ccbf14f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:03.007978 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cf77a0-78b9-4322-9278-6be2ccbf14f2 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:03.009161 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cf77a0-78b9-4322-9278-6be2ccbf14f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:03.009161 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cf77a0-78b9-4322-9278-6be2ccbf14f2 None admin] PATH_INFO: `/v3/services` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:03.009161 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cf77a0-78b9-4322-9278-6be2ccbf14f2 None admin] RBAC: Adding query filter params (name=image) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:02:03.009448 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cf77a0-78b9-4322-9278-6be2ccbf14f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:03.010791 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cf77a0-78b9-4322-9278-6be2ccbf14f2 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:03.015696 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 532/1048] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:02:02 2022] GET /identity/v3/services?name=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:02:03.016875 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83413760-34ff-4fc7-89a1-5b950dc1c1f5 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:03.017156 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83413760-34ff-4fc7-89a1-5b950dc1c1f5 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:03.017428 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83413760-34ff-4fc7-89a1-5b950dc1c1f5 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:03.019002 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 533/1049] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:02:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:02:03.021495 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83413760-34ff-4fc7-89a1-5b950dc1c1f5 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:03.032759 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04373c01-abd0-4306-9d1f-1324c865eda2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:03.033373 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04373c01-abd0-4306-9d1f-1324c865eda2 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:03.033657 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04373c01-abd0-4306-9d1f-1324c865eda2 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:03.033934 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04373c01-abd0-4306-9d1f-1324c865eda2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:03.034695 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04373c01-abd0-4306-9d1f-1324c865eda2 None admin] RBAC: Adding query filter params (role.id=b8d76e033b4b43ccbe4e580f17be8cc9, scope.project.id=9fd4be43faec4e4191e0e4ec4a4588c3, user.id=0cd171bf4df94f8fab397e42835feeab) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:02:03.035005 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04373c01-abd0-4306-9d1f-1324c865eda2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:03.036436 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04373c01-abd0-4306-9d1f-1324c865eda2 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:03.041999 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 517/1050] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Aug 25 03:02:02 2022] GET /identity/v3/role_assignments?user.id=0cd171bf4df94f8fab397e42835feeab&scope.project.id=9fd4be43faec4e4191e0e4ec4a4588c3&role.id=b8d76e033b4b43ccbe4e580f17be8cc9 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:02:03.043591 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138bc68c-d9f0-43be-b1d2-bcc9ffeee326 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:03.043882 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138bc68c-d9f0-43be-b1d2-bcc9ffeee326 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:03.044216 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138bc68c-d9f0-43be-b1d2-bcc9ffeee326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:03.058731 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:03.073986 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-138bc68c-d9f0-43be-b1d2-bcc9ffeee326 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:02:03.075271 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-138bc68c-d9f0-43be-b1d2-bcc9ffeee326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:02:03.087842 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:03.138711 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83413760-34ff-4fc7-89a1-5b950dc1c1f5 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:03.160704 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 518/1051] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:02:03 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 03:02:03.177401 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c6378e-a8dc-48a5-b256-cb16ed1383c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:03.178300 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c6378e-a8dc-48a5-b256-cb16ed1383c7 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:03.178576 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c6378e-a8dc-48a5-b256-cb16ed1383c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:03.178849 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c6378e-a8dc-48a5-b256-cb16ed1383c7 None admin] PATH_INFO: `/v3/services` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:03.179483 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c6378e-a8dc-48a5-b256-cb16ed1383c7 None admin] RBAC: Adding query filter params (type=image) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:02:03.179782 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c6378e-a8dc-48a5-b256-cb16ed1383c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:03.181139 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c6378e-a8dc-48a5-b256-cb16ed1383c7 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:03.186075 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 534/1052] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:02:03 2022] GET /identity/v3/services?type=image => generated 350 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:02:03.192858 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138bc68c-d9f0-43be-b1d2-bcc9ffeee326 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:03.292802 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138bc68c-d9f0-43be-b1d2-bcc9ffeee326 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:03.323478 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:03.329340 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7bc8b7-6474-4985-bfa3-d685ea03a52e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:03.330005 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7bc8b7-6474-4985-bfa3-d685ea03a52e None admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:03.330305 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7bc8b7-6474-4985-bfa3-d685ea03a52e None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:03.330584 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7bc8b7-6474-4985-bfa3-d685ea03a52e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:03.331252 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7bc8b7-6474-4985-bfa3-d685ea03a52e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:03.332620 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7bc8b7-6474-4985-bfa3-d685ea03a52e None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:03.434500 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 519/1053] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:02:03 2022] POST /identity/v3/endpoints => generated 327 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:02:03.437520 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c6378e-a8dc-48a5-b256-cb16ed1383c7 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:03.549691 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c6378e-a8dc-48a5-b256-cb16ed1383c7 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:03.571272 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:03.597421 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dce5df-e33b-4242-a844-472b7759e351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:03.598066 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dce5df-e33b-4242-a844-472b7759e351 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:03.598630 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dce5df-e33b-4242-a844-472b7759e351 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:03.598909 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dce5df-e33b-4242-a844-472b7759e351 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:03.610839 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dce5df-e33b-4242-a844-472b7759e351 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:02:03.615497 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-d9dce5df-e33b-4242-a844-472b7759e351 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 03:02:03.615497 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:02:03.615497 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:02:03.615497 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:02:03.615497 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:02:03.615497 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:02:03.615497 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:02:03.615497 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:02:03.615497 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:02:03.615497 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:02:03.615497 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:02:03.615497 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:02:03.615497 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:02:03.615497 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:02:03.615497 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:02:03.615497 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:02:03.616449 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:02:03.616449 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:02:03.616449 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:02:03.616449 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:02:03.616449 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:02:03.616449 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:02:03.616449 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:02:03.616449 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:02:03.616449 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:02:03.616449 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:02:03.616449 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:02:03.616449 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:02:03.616449 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:02:03.616449 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:02:03.616449 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:02:03.616449 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:02:03.617296 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:02:03.617296 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:02:03.617296 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:02:03.617296 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:02:03.617296 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:02:03.617296 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:02:03.617296 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:02:03.617296 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:02:03.617296 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:02:03.617296 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:02:03.617296 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 03:02:03.617296 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:02:03.617296 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 535/1054] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Aug 25 03:02:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:02:03.622727 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7bc8b7-6474-4985-bfa3-d685ea03a52e None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:03.727505 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7bc8b7-6474-4985-bfa3-d685ea03a52e None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:03.763005 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9194bdf3-520e-4f62-a90b-a642446c1af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:03.763619 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9194bdf3-520e-4f62-a90b-a642446c1af0 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:03.763900 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9194bdf3-520e-4f62-a90b-a642446c1af0 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:03.764174 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9194bdf3-520e-4f62-a90b-a642446c1af0 None admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:03.764952 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9194bdf3-520e-4f62-a90b-a642446c1af0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:02:03.765251 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9194bdf3-520e-4f62-a90b-a642446c1af0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:03.766650 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9194bdf3-520e-4f62-a90b-a642446c1af0 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:03.775449 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 520/1055] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Aug 25 03:02:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:02:03.781328 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dce5df-e33b-4242-a844-472b7759e351 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:03.888225 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dce5df-e33b-4242-a844-472b7759e351 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:03.925772 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a35a88-e4ec-477e-9828-ba8a0a1e8f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:03.926433 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a35a88-e4ec-477e-9828-ba8a0a1e8f41 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:03.926715 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a35a88-e4ec-477e-9828-ba8a0a1e8f41 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:03.926992 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a35a88-e4ec-477e-9828-ba8a0a1e8f41 None admin] PATH_INFO: `/v3/users/b7f8df8b323148828fda1dc23f5a9d55` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:03.941035 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a35a88-e4ec-477e-9828-ba8a0a1e8f41 None admin] RBAC: Authorizing `identity:get_user(user_id=b7f8df8b323148828fda1dc23f5a9d55)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:03.942497 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a35a88-e4ec-477e-9828-ba8a0a1e8f41 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:03.944120 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 536/1056] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Aug 25 03:02:03 2022] GET /identity/v3/users/b7f8df8b323148828fda1dc23f5a9d55 => generated 285 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:02:03.950136 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9194bdf3-520e-4f62-a90b-a642446c1af0 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:04.001319 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:04.051008 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9194bdf3-520e-4f62-a90b-a642446c1af0 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:04.083864 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8837cc7-6d09-4ed9-bdd4-e28ddf3478d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:04.084479 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8837cc7-6d09-4ed9-bdd4-e28ddf3478d8 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:04.084760 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8837cc7-6d09-4ed9-bdd4-e28ddf3478d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:04.085034 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8837cc7-6d09-4ed9-bdd4-e28ddf3478d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:04.153948 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8837cc7-6d09-4ed9-bdd4-e28ddf3478d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:04.155913 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8837cc7-6d09-4ed9-bdd4-e28ddf3478d8 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:04.202295 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:04.256501 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 521/1057] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Aug 25 03:02:03 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 25 03:02:04.257682 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a35a88-e4ec-477e-9828-ba8a0a1e8f41 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:04.291219 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:04.360628 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a35a88-e4ec-477e-9828-ba8a0a1e8f41 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:04.396796 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62e39ae-332b-43cc-ab60-10c914a2a968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:04.397426 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62e39ae-332b-43cc-ab60-10c914a2a968 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:04.397704 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62e39ae-332b-43cc-ab60-10c914a2a968 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:04.397973 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62e39ae-332b-43cc-ab60-10c914a2a968 None admin] PATH_INFO: `/v3/projects/4f4073ff732c4d429a6801e8dd46f752` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:04.399097 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62e39ae-332b-43cc-ab60-10c914a2a968 None admin] RBAC: Authorizing `identity:get_project(project_id=4f4073ff732c4d429a6801e8dd46f752)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:04.400511 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62e39ae-332b-43cc-ab60-10c914a2a968 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:04.407993 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 537/1058] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Aug 25 03:02:04 2022] GET /identity/v3/projects/4f4073ff732c4d429a6801e8dd46f752 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:02:04.408896 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8837cc7-6d09-4ed9-bdd4-e28ddf3478d8 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:04.507634 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8837cc7-6d09-4ed9-bdd4-e28ddf3478d8 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:04.542508 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ccee61-c089-43ca-a716-0911ab61ed55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:04.543106 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ccee61-c089-43ca-a716-0911ab61ed55 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:04.543384 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ccee61-c089-43ca-a716-0911ab61ed55 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:04.543656 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ccee61-c089-43ca-a716-0911ab61ed55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:04.544380 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ccee61-c089-43ca-a716-0911ab61ed55 None admin] RBAC: Adding query filter params (role.id=62f19ea272b447ccaab1576cd9bf4fd9, scope.project.id=4f4073ff732c4d429a6801e8dd46f752, user.id=b7f8df8b323148828fda1dc23f5a9d55) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:02:04.544686 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ccee61-c089-43ca-a716-0911ab61ed55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:04.546153 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ccee61-c089-43ca-a716-0911ab61ed55 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:04.558582 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 522/1059] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Aug 25 03:02:04 2022] GET /identity/v3/role_assignments?user.id=b7f8df8b323148828fda1dc23f5a9d55&scope.project.id=4f4073ff732c4d429a6801e8dd46f752&role.id=62f19ea272b447ccaab1576cd9bf4fd9 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:02:04.974625 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4e1393-1448-4d8f-a0b1-3a0dd6ed9167 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:04.975147 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4e1393-1448-4d8f-a0b1-3a0dd6ed9167 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:04.975147 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4e1393-1448-4d8f-a0b1-3a0dd6ed9167 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:04.976691 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 538/1060] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:02:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:02:04.982104 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe364ede-5a27-4001-b719-78be999eca28 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:04.982472 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe364ede-5a27-4001-b719-78be999eca28 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:04.982709 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe364ede-5a27-4001-b719-78be999eca28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:05.001101 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-fe364ede-5a27-4001-b719-78be999eca28 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:02:05.002318 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-fe364ede-5a27-4001-b719-78be999eca28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:02:05.012133 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:05.074623 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 523/1061] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:02:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 03:02:05.341690 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4e1393-1448-4d8f-a0b1-3a0dd6ed9167 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:05.404075 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9355c74-590f-4c18-b5d9-6ce3f6befaf9 None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:05.404473 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9355c74-590f-4c18-b5d9-6ce3f6befaf9 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:05.404757 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9355c74-590f-4c18-b5d9-6ce3f6befaf9 None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:05.406838 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 524/1062] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:02:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 25 03:02:05.412206 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b923f8bf-32f8-4431-87fd-fe032a15588f None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:05.412743 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b923f8bf-32f8-4431-87fd-fe032a15588f None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:05.412743 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b923f8bf-32f8-4431-87fd-fe032a15588f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:05.430026 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-b923f8bf-32f8-4431-87fd-fe032a15588f None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:02:05.431513 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-b923f8bf-32f8-4431-87fd-fe032a15588f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:02:05.447730 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4e1393-1448-4d8f-a0b1-3a0dd6ed9167 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:05.449381 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:05.485122 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f26ae8-7c9c-48c9-8792-52f6c1048ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:05.485778 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f26ae8-7c9c-48c9-8792-52f6c1048ac5 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:05.486053 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f26ae8-7c9c-48c9-8792-52f6c1048ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:05.486344 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f26ae8-7c9c-48c9-8792-52f6c1048ac5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:05.498441 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f26ae8-7c9c-48c9-8792-52f6c1048ac5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:02:05.503650 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-84f26ae8-7c9c-48c9-8792-52f6c1048ac5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 25 03:02:05.503650 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:02:05.503650 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:02:05.503650 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:02:05.503650 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:02:05.503650 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:02:05.503650 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:02:05.503650 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:02:05.503650 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:02:05.503650 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:02:05.503650 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:02:05.503650 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:02:05.503650 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:02:05.503650 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:02:05.503650 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:02:05.503650 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:02:05.504697 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:02:05.504697 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:02:05.504697 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:02:05.504697 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:02:05.504697 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:02:05.504697 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:02:05.504697 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:02:05.504697 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:02:05.504697 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:02:05.504697 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:02:05.504697 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:02:05.504697 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:02:05.504697 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:02:05.504697 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:02:05.504697 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:02:05.504697 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:02:05.505540 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:02:05.505540 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:02:05.505540 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:02:05.505540 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:02:05.505540 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:02:05.505540 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:02:05.505540 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:02:05.505540 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:02:05.505540 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:02:05.505540 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:02:05.505540 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 25 03:02:05.505540 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:02:05.509695 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 539/1063] 10.222.0.58 () {56 vars in 1120 bytes} [Thu Aug 25 03:02:05 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:02:05.510559 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 525/1064] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:02:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 25 03:02:05.510980 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f26ae8-7c9c-48c9-8792-52f6c1048ac5 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:05.617585 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f26ae8-7c9c-48c9-8792-52f6c1048ac5 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:05.656816 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e3eaad-07aa-4aca-b4ff-5ec7133ef0c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:05.657431 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e3eaad-07aa-4aca-b4ff-5ec7133ef0c6 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:05.657713 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e3eaad-07aa-4aca-b4ff-5ec7133ef0c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:05.657989 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e3eaad-07aa-4aca-b4ff-5ec7133ef0c6 None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:05.658722 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e3eaad-07aa-4aca-b4ff-5ec7133ef0c6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:02:05.659029 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e3eaad-07aa-4aca-b4ff-5ec7133ef0c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:05.660417 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e3eaad-07aa-4aca-b4ff-5ec7133ef0c6 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:05.673154 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 540/1065] 10.222.0.58 () {56 vars in 1115 bytes} [Thu Aug 25 03:02:05 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:02:05.674776 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b923f8bf-32f8-4431-87fd-fe032a15588f None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:05.754545 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:05.778896 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e3eaad-07aa-4aca-b4ff-5ec7133ef0c6 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:05.780048 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b923f8bf-32f8-4431-87fd-fe032a15588f None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:05.804232 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:05.815491 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bdefcd-663a-4757-a8bb-0ea073f90954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:05.816161 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bdefcd-663a-4757-a8bb-0ea073f90954 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:05.816496 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bdefcd-663a-4757-a8bb-0ea073f90954 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:05.816770 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bdefcd-663a-4757-a8bb-0ea073f90954 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:05.827099 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bdefcd-663a-4757-a8bb-0ea073f90954 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:05.829385 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bdefcd-663a-4757-a8bb-0ea073f90954 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:05.843136 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-87bdefcd-663a-4757-a8bb-0ea073f90954 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:02:05.843136 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:02:05.843136 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:02:05.843136 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:02:05.843136 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:02:05.843136 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:02:05.843136 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:02:05.843136 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:02:05.843136 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:02:05.843136 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:02:05.843136 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:02:05.843136 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:02:05.843136 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 03:02:05.843136 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 03:02:05.843136 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 03:02:05.843136 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 03:02:05.844130 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:02:05.844130 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:02:05.844130 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:02:05.844130 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:02:05.844130 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:02:05.844130 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:02:05.844130 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:02:05.844130 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:02:05.844130 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:02:05.844130 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:02:05.844130 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:02:05.844130 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:02:05.844130 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:02:05.844130 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:02:05.844130 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:02:05.844130 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:02:05.844130 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 03:02:05.845586 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 03:02:05.845586 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:02:05.845586 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:02:05.845586 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 526/1066] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:02:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 03:02:05.849879 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bdefcd-663a-4757-a8bb-0ea073f90954 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:05.885614 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e3eaad-07aa-4aca-b4ff-5ec7133ef0c6 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:05.921516 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628765cc-d977-4918-920b-6f52c924da9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:05.922121 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628765cc-d977-4918-920b-6f52c924da9a None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:05.922424 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628765cc-d977-4918-920b-6f52c924da9a None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:05.922691 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628765cc-d977-4918-920b-6f52c924da9a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:05.954815 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bdefcd-663a-4757-a8bb-0ea073f90954 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:05.981522 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:05.992137 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628765cc-d977-4918-920b-6f52c924da9a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:05.992664 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb76a416-2ea2-45b2-82bc-e0b3603e093b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:05.992923 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb76a416-2ea2-45b2-82bc-e0b3603e093b None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:05.993196 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb76a416-2ea2-45b2-82bc-e0b3603e093b None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:05.993467 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb76a416-2ea2-45b2-82bc-e0b3603e093b None admin] PATH_INFO: `/v3/domains` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:05.993636 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628765cc-d977-4918-920b-6f52c924da9a None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:05.994117 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb76a416-2ea2-45b2-82bc-e0b3603e093b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:02:05.994608 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb76a416-2ea2-45b2-82bc-e0b3603e093b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:05.995831 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb76a416-2ea2-45b2-82bc-e0b3603e093b None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:06.013152 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 527/1067] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Aug 25 03:02:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:02:06.014335 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb76a416-2ea2-45b2-82bc-e0b3603e093b None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:06.029146 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:06.114559 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 541/1068] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Aug 25 03:02:05 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 25 03:02:06.120214 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628765cc-d977-4918-920b-6f52c924da9a None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:06.129314 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb76a416-2ea2-45b2-82bc-e0b3603e093b None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:06.167433 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c11d3d2-5fd7-4261-8439-0ab69c75a588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:06.168061 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c11d3d2-5fd7-4261-8439-0ab69c75a588 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:06.168341 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c11d3d2-5fd7-4261-8439-0ab69c75a588 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:06.168612 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c11d3d2-5fd7-4261-8439-0ab69c75a588 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:06.176991 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c11d3d2-5fd7-4261-8439-0ab69c75a588 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:06.178460 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c11d3d2-5fd7-4261-8439-0ab69c75a588 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:06.191195 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-5c11d3d2-5fd7-4261-8439-0ab69c75a588 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 25 03:02:06.191195 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:02:06.191195 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:02:06.191195 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:02:06.191195 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:02:06.191195 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:02:06.191195 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:02:06.191195 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:02:06.191195 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:02:06.191195 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:02:06.191195 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:02:06.191195 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:02:06.191195 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 25 03:02:06.191195 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 25 03:02:06.191195 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 25 03:02:06.191195 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 25 03:02:06.192186 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:02:06.192186 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:02:06.192186 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 25 03:02:06.192186 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 03:02:06.192186 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 25 03:02:06.192186 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 03:02:06.192186 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:02:06.192186 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:02:06.192186 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:02:06.192186 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:02:06.192186 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:02:06.192186 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:02:06.192186 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:02:06.192186 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:02:06.192186 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:02:06.192186 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:02:06.192186 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:02:06.192186 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:02:06.193159 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:02:06.193159 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:02:06.193159 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 25 03:02:06.193159 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 25 03:02:06.193159 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 25 03:02:06.193159 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 25 03:02:06.193159 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 25 03:02:06.193159 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 25 03:02:06.193159 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 25 03:02:06.193159 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:02:06.193159 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 528/1069] 10.222.0.58 () {56 vars in 1093 bytes} [Thu Aug 25 03:02:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:02:06.197813 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c11d3d2-5fd7-4261-8439-0ab69c75a588 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:06.224984 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628765cc-d977-4918-920b-6f52c924da9a None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:06.246936 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:06.251504 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:06.262297 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63ba848-e9b7-4ee2-98e0-29ad3639169b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:06.263004 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63ba848-e9b7-4ee2-98e0-29ad3639169b None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:06.263376 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63ba848-e9b7-4ee2-98e0-29ad3639169b None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:06.263719 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63ba848-e9b7-4ee2-98e0-29ad3639169b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:06.274196 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63ba848-e9b7-4ee2-98e0-29ad3639169b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:06.275745 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63ba848-e9b7-4ee2-98e0-29ad3639169b None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:06.290781 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-a63ba848-e9b7-4ee2-98e0-29ad3639169b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:02:06.290781 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:02:06.290781 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:02:06.290781 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:02:06.290781 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:02:06.290781 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:02:06.290781 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:02:06.290781 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:02:06.290781 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:02:06.290781 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:02:06.290781 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:02:06.290781 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:02:06.290781 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 03:02:06.290781 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 03:02:06.290781 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 03:02:06.290781 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 03:02:06.292760 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:02:06.292760 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:02:06.292760 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:02:06.292760 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:02:06.292760 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:02:06.292760 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:02:06.292760 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:02:06.292760 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:02:06.292760 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:02:06.292760 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:02:06.292760 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:02:06.292760 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:02:06.292760 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:02:06.292760 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:02:06.292760 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:02:06.292760 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:02:06.292760 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 03:02:06.293668 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 03:02:06.293668 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:02:06.293668 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:02:06.297686 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 542/1070] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:02:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 03:02:06.298956 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63ba848-e9b7-4ee2-98e0-29ad3639169b None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:06.308261 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c11d3d2-5fd7-4261-8439-0ab69c75a588 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:06.343995 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f879cf-7593-4306-a802-ed047364ca93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:06.344876 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f879cf-7593-4306-a802-ed047364ca93 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:06.345326 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f879cf-7593-4306-a802-ed047364ca93 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:06.345761 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f879cf-7593-4306-a802-ed047364ca93 None admin] PATH_INFO: `/v3/users` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:06.346522 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f879cf-7593-4306-a802-ed047364ca93 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:02:06.346829 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f879cf-7593-4306-a802-ed047364ca93 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:06.348259 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f879cf-7593-4306-a802-ed047364ca93 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:06.366982 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 529/1071] 10.222.0.58 () {56 vars in 1133 bytes} [Thu Aug 25 03:02:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:02:06.374826 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f879cf-7593-4306-a802-ed047364ca93 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:06.414985 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63ba848-e9b7-4ee2-98e0-29ad3639169b None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:06.451117 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d78b0c-1f52-43c5-99e2-04baa63aa023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:06.451731 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d78b0c-1f52-43c5-99e2-04baa63aa023 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:06.452282 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d78b0c-1f52-43c5-99e2-04baa63aa023 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:06.453156 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d78b0c-1f52-43c5-99e2-04baa63aa023 None admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:06.453527 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d78b0c-1f52-43c5-99e2-04baa63aa023 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:02:06.453736 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d78b0c-1f52-43c5-99e2-04baa63aa023 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:06.455109 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d78b0c-1f52-43c5-99e2-04baa63aa023 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:06.478522 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 543/1072] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Aug 25 03:02:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:02:06.485582 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f879cf-7593-4306-a802-ed047364ca93 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:06.487941 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d4d576-7abc-41c3-8ef0-eb3da58d79a0 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:06.488219 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d4d576-7abc-41c3-8ef0-eb3da58d79a0 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:06.488497 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d4d576-7abc-41c3-8ef0-eb3da58d79a0 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:06.489887 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 544/1073] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:02:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 25 03:02:06.490453 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:06.495688 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f15aa2-c7d7-4937-9627-6adda2aba687 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:06.495975 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f15aa2-c7d7-4937-9627-6adda2aba687 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:06.496256 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f15aa2-c7d7-4937-9627-6adda2aba687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:06.515955 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-c3f15aa2-c7d7-4937-9627-6adda2aba687 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:02:06.517564 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-c3f15aa2-c7d7-4937-9627-6adda2aba687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:02:06.525569 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2690df2-70a7-4fa9-9c56-2bea3a0b00fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:06.526393 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2690df2-70a7-4fa9-9c56-2bea3a0b00fe None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:06.527614 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2690df2-70a7-4fa9-9c56-2bea3a0b00fe None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:06.527889 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2690df2-70a7-4fa9-9c56-2bea3a0b00fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:06.539365 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2690df2-70a7-4fa9-9c56-2bea3a0b00fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:06.540856 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2690df2-70a7-4fa9-9c56-2bea3a0b00fe None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:06.555424 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:06.559234 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-e2690df2-70a7-4fa9-9c56-2bea3a0b00fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:02:06.559234 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:02:06.559234 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:02:06.559234 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:02:06.559234 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:02:06.559234 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:02:06.559234 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:02:06.559234 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:02:06.559234 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:02:06.559234 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:02:06.559234 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:02:06.559234 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:02:06.559234 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 03:02:06.559234 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 03:02:06.559234 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 03:02:06.559234 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 03:02:06.560222 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:02:06.560222 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:02:06.560222 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:02:06.560222 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:02:06.560222 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:02:06.560222 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:02:06.560222 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:02:06.560222 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:02:06.560222 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:02:06.560222 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:02:06.560222 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:02:06.560222 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:02:06.560222 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:02:06.560222 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:02:06.560222 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:02:06.560222 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:02:06.560222 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 03:02:06.561122 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 03:02:06.561122 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:02:06.561122 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:02:06.561122 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 530/1074] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:02:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:02:06.566439 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2690df2-70a7-4fa9-9c56-2bea3a0b00fe None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:06.602791 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 545/1075] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:02:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 25 03:02:06.682491 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2690df2-70a7-4fa9-9c56-2bea3a0b00fe None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:06.721837 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae9bd12-2739-4301-8454-2a67b14616e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:06.722818 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae9bd12-2739-4301-8454-2a67b14616e2 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:06.723102 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae9bd12-2739-4301-8454-2a67b14616e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:06.723376 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae9bd12-2739-4301-8454-2a67b14616e2 None admin] PATH_INFO: `/v3/domains` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:06.724056 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae9bd12-2739-4301-8454-2a67b14616e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:02:06.724359 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae9bd12-2739-4301-8454-2a67b14616e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:06.725721 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae9bd12-2739-4301-8454-2a67b14616e2 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:06.738393 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 531/1076] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Aug 25 03:02:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:02:06.744074 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f15aa2-c7d7-4937-9627-6adda2aba687 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:06.845426 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f15aa2-c7d7-4937-9627-6adda2aba687 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:06.875988 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae9bd12-2739-4301-8454-2a67b14616e2 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:06.877944 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db5dbc8-8dd1-4e15-9e7a-a0e74106add4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:06.878724 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db5dbc8-8dd1-4e15-9e7a-a0e74106add4 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:06.878881 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db5dbc8-8dd1-4e15-9e7a-a0e74106add4 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:06.879153 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db5dbc8-8dd1-4e15-9e7a-a0e74106add4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:06.885828 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db5dbc8-8dd1-4e15-9e7a-a0e74106add4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:06.887309 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db5dbc8-8dd1-4e15-9e7a-a0e74106add4 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:06.894068 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:06.898645 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-4db5dbc8-8dd1-4e15-9e7a-a0e74106add4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 03:02:06.898645 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:02:06.898645 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:02:06.898645 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:02:06.898645 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:02:06.898645 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:02:06.898645 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:02:06.898645 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:02:06.898645 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:02:06.898645 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:02:06.898645 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:02:06.898645 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:02:06.898645 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 25 03:02:06.898645 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 25 03:02:06.898645 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 25 03:02:06.899774 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 25 03:02:06.899774 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:02:06.899774 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:02:06.899774 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:02:06.899774 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:02:06.899774 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:02:06.899774 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:02:06.899774 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:02:06.899774 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:02:06.899774 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:02:06.899774 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:02:06.899774 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:02:06.899774 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:02:06.899774 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:02:06.899774 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:02:06.899774 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:02:06.899774 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:02:06.900708 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 25 03:02:06.900708 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 25 03:02:06.900708 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 25 03:02:06.900708 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 25 03:02:06.900708 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 25 03:02:06.900708 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 25 03:02:06.900708 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 03:02:06.900708 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:02:06.900708 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 546/1077] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Aug 25 03:02:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:02:06.906023 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db5dbc8-8dd1-4e15-9e7a-a0e74106add4 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:06.933233 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:06.975787 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae9bd12-2739-4301-8454-2a67b14616e2 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:07.005930 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db5dbc8-8dd1-4e15-9e7a-a0e74106add4 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:07.009843 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5de14c-39b9-496b-8515-54cc331a66da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:07.010510 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5de14c-39b9-496b-8515-54cc331a66da None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:07.010794 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5de14c-39b9-496b-8515-54cc331a66da None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:07.011070 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5de14c-39b9-496b-8515-54cc331a66da None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:07.021584 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5de14c-39b9-496b-8515-54cc331a66da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:02:07.026608 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-bf5de14c-39b9-496b-8515-54cc331a66da None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 03:02:07.026608 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:02:07.026608 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:02:07.026608 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:02:07.026608 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:02:07.026608 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:02:07.026608 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:02:07.026608 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:02:07.026608 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:02:07.026608 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:02:07.026608 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:02:07.026608 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:02:07.026608 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:02:07.026608 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:02:07.026608 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:02:07.026608 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:02:07.027731 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:02:07.027731 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:02:07.027731 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:02:07.027731 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:02:07.027731 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:02:07.027731 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:02:07.027731 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:02:07.027731 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:02:07.027731 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:02:07.027731 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:02:07.027731 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:02:07.027731 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:02:07.027731 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:02:07.027731 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:02:07.027731 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:02:07.027731 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:02:07.028661 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:02:07.028661 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:02:07.028661 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:02:07.028661 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:02:07.028661 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:02:07.028661 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:02:07.028661 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:02:07.028661 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:02:07.028661 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:02:07.028661 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:02:07.028661 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 03:02:07.028661 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:02:07.028661 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 532/1078] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Aug 25 03:02:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 03:02:07.034059 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5de14c-39b9-496b-8515-54cc331a66da None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:07.047451 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936b5e05-8b6b-4ac0-a593-5b04eab250eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:07.048454 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936b5e05-8b6b-4ac0-a593-5b04eab250eb None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:07.048941 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936b5e05-8b6b-4ac0-a593-5b04eab250eb None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:07.049413 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936b5e05-8b6b-4ac0-a593-5b04eab250eb None admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:07.050793 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936b5e05-8b6b-4ac0-a593-5b04eab250eb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:02:07.051298 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936b5e05-8b6b-4ac0-a593-5b04eab250eb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:07.052731 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936b5e05-8b6b-4ac0-a593-5b04eab250eb None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:07.064995 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 547/1079] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Aug 25 03:02:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:02:07.070834 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936b5e05-8b6b-4ac0-a593-5b04eab250eb None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:07.115378 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:07.137573 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5de14c-39b9-496b-8515-54cc331a66da None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:07.152885 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:07.167261 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936b5e05-8b6b-4ac0-a593-5b04eab250eb None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:07.171776 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d5070b-c1dc-4a33-8ce4-8b54347fcd08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:07.172375 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d5070b-c1dc-4a33-8ce4-8b54347fcd08 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:07.172650 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d5070b-c1dc-4a33-8ce4-8b54347fcd08 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:07.172921 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d5070b-c1dc-4a33-8ce4-8b54347fcd08 None admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:07.173573 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d5070b-c1dc-4a33-8ce4-8b54347fcd08 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:02:07.173871 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d5070b-c1dc-4a33-8ce4-8b54347fcd08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:07.175259 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d5070b-c1dc-4a33-8ce4-8b54347fcd08 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:07.183456 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 533/1080] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Aug 25 03:02:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:02:07.189645 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d5070b-c1dc-4a33-8ce4-8b54347fcd08 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:07.202076 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcf529c-7619-44ef-be4e-4c817bd2482a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:07.203457 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcf529c-7619-44ef-be4e-4c817bd2482a None admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:07.203748 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcf529c-7619-44ef-be4e-4c817bd2482a None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:07.204021 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcf529c-7619-44ef-be4e-4c817bd2482a None admin] PATH_INFO: `/v3/projects/9fd4be43faec4e4191e0e4ec4a4588c3/users/0cd171bf4df94f8fab397e42835feeab/roles/b8d76e033b4b43ccbe4e580f17be8cc9` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:07.206201 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcf529c-7619-44ef-be4e-4c817bd2482a None admin] RBAC: Authorizing `identity:create_grant(project_id=9fd4be43faec4e4191e0e4ec4a4588c3, user_id=0cd171bf4df94f8fab397e42835feeab, role_id=b8d76e033b4b43ccbe4e580f17be8cc9)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:07.207653 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcf529c-7619-44ef-be4e-4c817bd2482a None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:07.269550 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 548/1081] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Aug 25 03:02:07 2022] PUT /identity/v3/projects/9fd4be43faec4e4191e0e4ec4a4588c3/users/0cd171bf4df94f8fab397e42835feeab/roles/b8d76e033b4b43ccbe4e580f17be8cc9 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:02:07.287708 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d5070b-c1dc-4a33-8ce4-8b54347fcd08 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:07.324136 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79833422-91a9-4ad2-bb6c-55650c41879e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:07.324784 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79833422-91a9-4ad2-bb6c-55650c41879e None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:07.325067 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79833422-91a9-4ad2-bb6c-55650c41879e None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:07.325342 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79833422-91a9-4ad2-bb6c-55650c41879e None admin] PATH_INFO: `/v3/users/b7f8df8b323148828fda1dc23f5a9d55` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:07.326540 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79833422-91a9-4ad2-bb6c-55650c41879e None admin] RBAC: Authorizing `identity:get_user(user_id=b7f8df8b323148828fda1dc23f5a9d55)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:07.329007 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79833422-91a9-4ad2-bb6c-55650c41879e None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:07.330717 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 534/1082] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Aug 25 03:02:07 2022] GET /identity/v3/users/b7f8df8b323148828fda1dc23f5a9d55 => generated 285 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:02:07.336590 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcf529c-7619-44ef-be4e-4c817bd2482a None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:07.440482 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcf529c-7619-44ef-be4e-4c817bd2482a None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:07.473684 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87561163-78ab-4efb-8d94-db8cc2654efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:07.474334 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87561163-78ab-4efb-8d94-db8cc2654efa None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:07.474612 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87561163-78ab-4efb-8d94-db8cc2654efa None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:07.474883 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87561163-78ab-4efb-8d94-db8cc2654efa None admin] PATH_INFO: `/v3/projects/4f4073ff732c4d429a6801e8dd46f752` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:07.475969 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87561163-78ab-4efb-8d94-db8cc2654efa None admin] RBAC: Authorizing `identity:get_project(project_id=4f4073ff732c4d429a6801e8dd46f752)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:07.477401 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87561163-78ab-4efb-8d94-db8cc2654efa None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:07.479122 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 549/1083] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Aug 25 03:02:07 2022] GET /identity/v3/projects/4f4073ff732c4d429a6801e8dd46f752 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:02:07.485478 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79833422-91a9-4ad2-bb6c-55650c41879e None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:07.494716 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:07.583902 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79833422-91a9-4ad2-bb6c-55650c41879e None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:07.616200 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9215ec4b-7cfa-443b-8178-304f8baf859b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:07.616852 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9215ec4b-7cfa-443b-8178-304f8baf859b None admin] REQUEST_METHOD: `PUT` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:07.617128 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9215ec4b-7cfa-443b-8178-304f8baf859b None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:07.617399 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9215ec4b-7cfa-443b-8178-304f8baf859b None admin] PATH_INFO: `/v3/projects/4f4073ff732c4d429a6801e8dd46f752/users/b7f8df8b323148828fda1dc23f5a9d55/roles/62f19ea272b447ccaab1576cd9bf4fd9` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:07.619278 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9215ec4b-7cfa-443b-8178-304f8baf859b None admin] RBAC: Authorizing `identity:create_grant(project_id=4f4073ff732c4d429a6801e8dd46f752, user_id=b7f8df8b323148828fda1dc23f5a9d55, role_id=62f19ea272b447ccaab1576cd9bf4fd9)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:07.620706 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9215ec4b-7cfa-443b-8178-304f8baf859b None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:07.697900 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 535/1084] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Aug 25 03:02:07 2022] PUT /identity/v3/projects/4f4073ff732c4d429a6801e8dd46f752/users/b7f8df8b323148828fda1dc23f5a9d55/roles/62f19ea272b447ccaab1576cd9bf4fd9 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:02:09.164126 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0bac72-1007-4640-8bbe-09a8cf79b6e4 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:09.164690 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0bac72-1007-4640-8bbe-09a8cf79b6e4 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:09.164690 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0bac72-1007-4640-8bbe-09a8cf79b6e4 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:09.166034 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 550/1085] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:02:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:02:09.171501 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379672b1-8826-4954-9f1b-dceb6f5be4eb None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:09.171888 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379672b1-8826-4954-9f1b-dceb6f5be4eb None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:09.172054 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379672b1-8826-4954-9f1b-dceb6f5be4eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:09.190563 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-379672b1-8826-4954-9f1b-dceb6f5be4eb None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:02:09.191751 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-379672b1-8826-4954-9f1b-dceb6f5be4eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:02:09.250035 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:09.257259 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 536/1086] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:02:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 03:02:09.526755 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0bac72-1007-4640-8bbe-09a8cf79b6e4 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:09.542484 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:09.573410 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec3f9ed-b5c6-40d3-bae3-bf103d3a7a99 None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:09.573695 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec3f9ed-b5c6-40d3-bae3-bf103d3a7a99 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:09.573975 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec3f9ed-b5c6-40d3-bae3-bf103d3a7a99 None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:09.575360 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 537/1087] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:02:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 25 03:02:09.582252 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47949d8-91bd-4d37-bcbf-149a483fbfd6 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:09.582715 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47949d8-91bd-4d37-bcbf-149a483fbfd6 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:09.582999 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47949d8-91bd-4d37-bcbf-149a483fbfd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:09.600485 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-d47949d8-91bd-4d37-bcbf-149a483fbfd6 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:02:09.601680 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-d47949d8-91bd-4d37-bcbf-149a483fbfd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:02:09.631530 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0bac72-1007-4640-8bbe-09a8cf79b6e4 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:09.664599 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838faaa9-fdec-4b09-846e-8a77544ae56f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:09.665231 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838faaa9-fdec-4b09-846e-8a77544ae56f None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:09.665513 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838faaa9-fdec-4b09-846e-8a77544ae56f None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:09.665786 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838faaa9-fdec-4b09-846e-8a77544ae56f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:09.667485 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 538/1088] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:02:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 03:02:09.676623 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838faaa9-fdec-4b09-846e-8a77544ae56f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:02:09.681225 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-838faaa9-fdec-4b09-846e-8a77544ae56f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 25 03:02:09.681225 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:02:09.681225 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:02:09.681225 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:02:09.681225 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:02:09.681225 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:02:09.681225 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:02:09.681225 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:02:09.681225 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:02:09.681225 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:02:09.681225 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:02:09.681225 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:02:09.681225 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:02:09.681225 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:02:09.681225 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:02:09.681225 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:02:09.682341 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:02:09.682341 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:02:09.682341 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:02:09.682341 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:02:09.682341 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:02:09.682341 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:02:09.682341 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:02:09.682341 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:02:09.682341 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:02:09.682341 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:02:09.682341 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:02:09.682341 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:02:09.682341 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:02:09.682341 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:02:09.682341 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:02:09.682341 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:02:09.683206 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:02:09.683206 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:02:09.683206 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:02:09.683206 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:02:09.683206 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:02:09.683206 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:02:09.683206 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:02:09.683206 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:02:09.683206 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:02:09.683206 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:02:09.683206 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 25 03:02:09.683206 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:02:09.683206 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 551/1089] 10.222.0.58 () {56 vars in 1120 bytes} [Thu Aug 25 03:02:09 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:02:09.688806 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47949d8-91bd-4d37-bcbf-149a483fbfd6 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:09.787300 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47949d8-91bd-4d37-bcbf-149a483fbfd6 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:09.815334 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:09.823743 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45692c5b-eaa3-4db3-9822-fba7db0764a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:09.824354 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45692c5b-eaa3-4db3-9822-fba7db0764a9 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:09.824631 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45692c5b-eaa3-4db3-9822-fba7db0764a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:09.824905 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45692c5b-eaa3-4db3-9822-fba7db0764a9 None admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:09.825572 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45692c5b-eaa3-4db3-9822-fba7db0764a9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:02:09.825870 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45692c5b-eaa3-4db3-9822-fba7db0764a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:09.827302 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45692c5b-eaa3-4db3-9822-fba7db0764a9 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:09.835332 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 539/1090] 10.222.0.58 () {56 vars in 1115 bytes} [Thu Aug 25 03:02:09 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:02:09.841000 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838faaa9-fdec-4b09-846e-8a77544ae56f None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:09.912924 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:09.938175 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838faaa9-fdec-4b09-846e-8a77544ae56f None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:09.944665 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45692c5b-eaa3-4db3-9822-fba7db0764a9 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:09.972816 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218fffa9-382d-4a81-9cc8-7e7655921324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:09.973856 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218fffa9-382d-4a81-9cc8-7e7655921324 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:09.973856 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218fffa9-382d-4a81-9cc8-7e7655921324 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:09.974026 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218fffa9-382d-4a81-9cc8-7e7655921324 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:09.984995 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218fffa9-382d-4a81-9cc8-7e7655921324 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:09.986538 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218fffa9-382d-4a81-9cc8-7e7655921324 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:10.000800 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-218fffa9-382d-4a81-9cc8-7e7655921324 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:02:10.000800 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:02:10.000800 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:02:10.000800 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:02:10.000800 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:02:10.000800 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:02:10.000800 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:02:10.000800 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:02:10.000800 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:02:10.000800 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:02:10.000800 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:02:10.000800 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:02:10.000800 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 03:02:10.000800 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 03:02:10.000800 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 03:02:10.000800 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 03:02:10.003366 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:02:10.003366 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:02:10.003366 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:02:10.003366 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:02:10.003366 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:02:10.003366 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:02:10.003366 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:02:10.003366 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:02:10.003366 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:02:10.003366 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:02:10.003366 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:02:10.003366 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:02:10.003366 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:02:10.003366 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:02:10.003366 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:02:10.003366 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:02:10.003366 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 03:02:10.004508 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 03:02:10.004508 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:02:10.004508 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:02:10.004508 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 552/1091] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:02:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 03:02:10.007724 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218fffa9-382d-4a81-9cc8-7e7655921324 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:10.047621 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45692c5b-eaa3-4db3-9822-fba7db0764a9 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:10.159104 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218fffa9-382d-4a81-9cc8-7e7655921324 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:10.160960 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b2dac2-6fb9-4288-b833-4a821ac007f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:10.161594 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b2dac2-6fb9-4288-b833-4a821ac007f3 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:10.161875 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b2dac2-6fb9-4288-b833-4a821ac007f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:10.162149 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b2dac2-6fb9-4288-b833-4a821ac007f3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:10.175404 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b2dac2-6fb9-4288-b833-4a821ac007f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:02:10.180141 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-87b2dac2-6fb9-4288-b833-4a821ac007f3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 03:02:10.180141 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:02:10.180141 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:02:10.180141 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:02:10.180141 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:02:10.180141 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:02:10.180141 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:02:10.180141 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:02:10.180141 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:02:10.180141 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:02:10.180141 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:02:10.180141 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:02:10.180141 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:02:10.180141 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:02:10.180141 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:02:10.180141 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:02:10.181396 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:02:10.181396 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:02:10.181396 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:02:10.181396 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:02:10.181396 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:02:10.181396 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:02:10.181396 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:02:10.181396 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:02:10.181396 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:02:10.181396 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:02:10.181396 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:02:10.181396 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:02:10.181396 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:02:10.181396 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:02:10.181396 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:02:10.181396 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:02:10.182264 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:02:10.182264 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:02:10.182264 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:02:10.182264 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:02:10.182264 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:02:10.182264 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:02:10.182264 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:02:10.182264 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:02:10.182264 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:02:10.182264 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:02:10.182264 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 03:02:10.182264 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:02:10.182264 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 540/1092] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Aug 25 03:02:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 03:02:10.186836 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:10.188391 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b2dac2-6fb9-4288-b833-4a821ac007f3 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:10.195533 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c89f63-b7c3-4481-893e-ca1704d82a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:10.196140 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c89f63-b7c3-4481-893e-ca1704d82a87 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:10.196419 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c89f63-b7c3-4481-893e-ca1704d82a87 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:10.196713 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c89f63-b7c3-4481-893e-ca1704d82a87 None admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:10.197364 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c89f63-b7c3-4481-893e-ca1704d82a87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:02:10.197664 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c89f63-b7c3-4481-893e-ca1704d82a87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:10.199127 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c89f63-b7c3-4481-893e-ca1704d82a87 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:10.211840 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 553/1093] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Aug 25 03:02:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:02:10.217670 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c89f63-b7c3-4481-893e-ca1704d82a87 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:10.238110 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:10.287645 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b2dac2-6fb9-4288-b833-4a821ac007f3 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:10.314967 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c89f63-b7c3-4481-893e-ca1704d82a87 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:10.321005 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b701c453-ca65-44e2-aa14-7dfd652bfbde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:10.321607 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b701c453-ca65-44e2-aa14-7dfd652bfbde None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:10.321886 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b701c453-ca65-44e2-aa14-7dfd652bfbde None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:10.322164 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b701c453-ca65-44e2-aa14-7dfd652bfbde None admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:10.322849 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b701c453-ca65-44e2-aa14-7dfd652bfbde None admin] RBAC: Adding query filter params (name=admin) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:02:10.323147 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b701c453-ca65-44e2-aa14-7dfd652bfbde None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:10.324726 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b701c453-ca65-44e2-aa14-7dfd652bfbde None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:10.333252 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 541/1094] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Aug 25 03:02:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:02:10.338914 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b701c453-ca65-44e2-aa14-7dfd652bfbde None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:10.349655 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74451a6c-61b2-4410-aa38-d8906a0d7ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:10.350305 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74451a6c-61b2-4410-aa38-d8906a0d7ee0 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:10.350598 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74451a6c-61b2-4410-aa38-d8906a0d7ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:10.350873 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74451a6c-61b2-4410-aa38-d8906a0d7ee0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:10.357796 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74451a6c-61b2-4410-aa38-d8906a0d7ee0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:10.359811 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74451a6c-61b2-4410-aa38-d8906a0d7ee0 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:10.371028 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-74451a6c-61b2-4410-aa38-d8906a0d7ee0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 25 03:02:10.371028 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:02:10.371028 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:02:10.371028 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:02:10.371028 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:02:10.371028 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:02:10.371028 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:02:10.371028 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:02:10.371028 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:02:10.371028 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:02:10.371028 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:02:10.371028 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:02:10.371028 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 25 03:02:10.371028 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 25 03:02:10.371028 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 25 03:02:10.371028 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 25 03:02:10.372102 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:02:10.372102 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:02:10.372102 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 25 03:02:10.372102 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 03:02:10.372102 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 25 03:02:10.372102 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 03:02:10.372102 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:02:10.372102 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:02:10.372102 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:02:10.372102 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:02:10.372102 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:02:10.372102 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:02:10.372102 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:02:10.372102 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:02:10.372102 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:02:10.372102 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:02:10.372102 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:02:10.372102 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:02:10.373564 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:02:10.373564 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:02:10.373564 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 25 03:02:10.373564 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 25 03:02:10.373564 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 25 03:02:10.373564 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 25 03:02:10.373564 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 25 03:02:10.373564 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 25 03:02:10.373564 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 25 03:02:10.373564 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:02:10.373564 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 554/1095] 10.222.0.58 () {56 vars in 1093 bytes} [Thu Aug 25 03:02:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 03:02:10.377790 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74451a6c-61b2-4410-aa38-d8906a0d7ee0 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:10.423805 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:10.441266 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b701c453-ca65-44e2-aa14-7dfd652bfbde None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:10.451420 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:10.475305 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc06152b-100a-4b45-aacf-6d8e717aed9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:10.475937 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc06152b-100a-4b45-aacf-6d8e717aed9b None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:10.476220 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc06152b-100a-4b45-aacf-6d8e717aed9b None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:10.476499 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc06152b-100a-4b45-aacf-6d8e717aed9b None admin] PATH_INFO: `/v3/users/b7f8df8b323148828fda1dc23f5a9d55` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:10.476855 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74451a6c-61b2-4410-aa38-d8906a0d7ee0 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:10.477642 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc06152b-100a-4b45-aacf-6d8e717aed9b None admin] RBAC: Authorizing `identity:get_user(user_id=b7f8df8b323148828fda1dc23f5a9d55)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:10.479133 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc06152b-100a-4b45-aacf-6d8e717aed9b None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:10.482436 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 542/1096] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Aug 25 03:02:10 2022] GET /identity/v3/users/b7f8df8b323148828fda1dc23f5a9d55 => generated 285 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:02:10.486744 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc06152b-100a-4b45-aacf-6d8e717aed9b None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:10.516138 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f29454-b441-4804-b4c0-385c520f145c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:10.516754 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f29454-b441-4804-b4c0-385c520f145c None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:10.517032 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f29454-b441-4804-b4c0-385c520f145c None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:10.517307 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f29454-b441-4804-b4c0-385c520f145c None admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:10.518036 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f29454-b441-4804-b4c0-385c520f145c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:02:10.518381 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f29454-b441-4804-b4c0-385c520f145c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:10.521695 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f29454-b441-4804-b4c0-385c520f145c None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:10.539922 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 555/1097] 10.222.0.58 () {56 vars in 1133 bytes} [Thu Aug 25 03:02:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:02:10.545670 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f29454-b441-4804-b4c0-385c520f145c None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:10.587156 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc06152b-100a-4b45-aacf-6d8e717aed9b None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:10.620148 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc19532-66ea-417a-b7e2-892661e1a58c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:10.620759 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc19532-66ea-417a-b7e2-892661e1a58c None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:10.621038 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc19532-66ea-417a-b7e2-892661e1a58c None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:10.621309 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc19532-66ea-417a-b7e2-892661e1a58c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:10.647839 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f29454-b441-4804-b4c0-385c520f145c None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:10.650231 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:10.669014 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:10.682409 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b80fd1-4e26-4add-bf01-13d3fc5b5ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:10.683030 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b80fd1-4e26-4add-bf01-13d3fc5b5ff0 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:10.683312 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b80fd1-4e26-4add-bf01-13d3fc5b5ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:10.683587 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b80fd1-4e26-4add-bf01-13d3fc5b5ff0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:10.690019 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc19532-66ea-417a-b7e2-892661e1a58c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:10.691519 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc19532-66ea-417a-b7e2-892661e1a58c None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:10.750085 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b80fd1-4e26-4add-bf01-13d3fc5b5ff0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:10.751746 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b80fd1-4e26-4add-bf01-13d3fc5b5ff0 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:10.789412 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 543/1098] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Aug 25 03:02:10 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 25 03:02:10.795228 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc19532-66ea-417a-b7e2-892661e1a58c None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:10.857099 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 556/1099] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Aug 25 03:02:10 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 25 03:02:10.858082 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b80fd1-4e26-4add-bf01-13d3fc5b5ff0 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:10.860816 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:10.872627 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:10.894085 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc19532-66ea-417a-b7e2-892661e1a58c None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:10.926666 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8e7025-cb6b-40f9-9025-d1dd657773e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:10.927287 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8e7025-cb6b-40f9-9025-d1dd657773e9 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:10.927566 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8e7025-cb6b-40f9-9025-d1dd657773e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:10.927839 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8e7025-cb6b-40f9-9025-d1dd657773e9 None admin] PATH_INFO: `/v3/projects/4f4073ff732c4d429a6801e8dd46f752` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:10.928965 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8e7025-cb6b-40f9-9025-d1dd657773e9 None admin] RBAC: Authorizing `identity:get_project(project_id=4f4073ff732c4d429a6801e8dd46f752)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:10.930455 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8e7025-cb6b-40f9-9025-d1dd657773e9 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:10.932061 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 544/1100] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Aug 25 03:02:10 2022] GET /identity/v3/projects/4f4073ff732c4d429a6801e8dd46f752 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:02:10.938523 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8e7025-cb6b-40f9-9025-d1dd657773e9 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:10.955921 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b80fd1-4e26-4add-bf01-13d3fc5b5ff0 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:10.989647 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6de281-38a0-42fc-b50a-eb47315065de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:10.990302 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6de281-38a0-42fc-b50a-eb47315065de None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:10.990582 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6de281-38a0-42fc-b50a-eb47315065de None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:10.990858 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6de281-38a0-42fc-b50a-eb47315065de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:11.001672 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6de281-38a0-42fc-b50a-eb47315065de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:11.003211 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6de281-38a0-42fc-b50a-eb47315065de None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:11.017471 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-dd6de281-38a0-42fc-b50a-eb47315065de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:02:11.017471 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:02:11.017471 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:02:11.017471 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:02:11.017471 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:02:11.017471 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:02:11.017471 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:02:11.017471 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:02:11.017471 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:02:11.017471 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:02:11.017471 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:02:11.017471 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:02:11.017471 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 03:02:11.017471 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 03:02:11.017471 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 03:02:11.017471 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 03:02:11.019028 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:02:11.019028 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:02:11.019028 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:02:11.019028 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:02:11.019028 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:02:11.019028 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:02:11.019028 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:02:11.019028 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:02:11.019028 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:02:11.019028 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:02:11.019028 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:02:11.019028 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:02:11.019028 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:02:11.019028 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:02:11.019028 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:02:11.019028 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:02:11.019028 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 03:02:11.020215 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 03:02:11.020215 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:02:11.020215 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:02:11.020215 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 557/1101] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:02:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 03:02:11.024091 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6de281-38a0-42fc-b50a-eb47315065de None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:11.040712 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8e7025-cb6b-40f9-9025-d1dd657773e9 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:11.075578 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0f7b60-e469-4876-95b2-8d0540756538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:11.076181 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0f7b60-e469-4876-95b2-8d0540756538 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:11.076465 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0f7b60-e469-4876-95b2-8d0540756538 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:11.076748 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0f7b60-e469-4876-95b2-8d0540756538 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:11.078016 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0f7b60-e469-4876-95b2-8d0540756538 None admin] RBAC: Adding query filter params (role.id=62f19ea272b447ccaab1576cd9bf4fd9, scope.project.id=4f4073ff732c4d429a6801e8dd46f752, user.id=b7f8df8b323148828fda1dc23f5a9d55) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:02:11.078604 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0f7b60-e469-4876-95b2-8d0540756538 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:11.079017 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:11.080040 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0f7b60-e469-4876-95b2-8d0540756538 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:11.085815 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 545/1102] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Aug 25 03:02:10 2022] GET /identity/v3/role_assignments?user.id=b7f8df8b323148828fda1dc23f5a9d55&scope.project.id=4f4073ff732c4d429a6801e8dd46f752&role.id=62f19ea272b447ccaab1576cd9bf4fd9 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:02:11.126784 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6de281-38a0-42fc-b50a-eb47315065de None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:11.162409 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d348ed-bcaf-400d-a5b6-78496616a2fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:11.163023 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d348ed-bcaf-400d-a5b6-78496616a2fe None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:11.163300 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d348ed-bcaf-400d-a5b6-78496616a2fe None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:11.163570 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d348ed-bcaf-400d-a5b6-78496616a2fe None admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:11.164218 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d348ed-bcaf-400d-a5b6-78496616a2fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:02:11.164515 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d348ed-bcaf-400d-a5b6-78496616a2fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:11.165896 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d348ed-bcaf-400d-a5b6-78496616a2fe None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:11.179311 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 558/1103] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Aug 25 03:02:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:02:11.185176 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0f7b60-e469-4876-95b2-8d0540756538 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:11.198954 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:11.288214 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0f7b60-e469-4876-95b2-8d0540756538 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:11.321427 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aee47e3-bebe-497e-921e-a766af1a26af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:11.322053 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aee47e3-bebe-497e-921e-a766af1a26af None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:11.322364 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aee47e3-bebe-497e-921e-a766af1a26af None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:11.322642 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aee47e3-bebe-497e-921e-a766af1a26af None admin] PATH_INFO: `/v3/projects/service` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:11.329387 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aee47e3-bebe-497e-921e-a766af1a26af None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:11.330884 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aee47e3-bebe-497e-921e-a766af1a26af None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:11.342475 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-8aee47e3-bebe-497e-921e-a766af1a26af None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 03:02:11.342475 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:02:11.342475 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:02:11.342475 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:02:11.342475 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:02:11.342475 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:02:11.342475 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:02:11.342475 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:02:11.342475 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:02:11.342475 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:02:11.342475 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:02:11.342475 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:02:11.342475 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 25 03:02:11.342475 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 25 03:02:11.342475 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 25 03:02:11.343352 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 25 03:02:11.343352 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:02:11.343352 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:02:11.343352 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:02:11.343352 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:02:11.343352 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:02:11.343352 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:02:11.343352 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:02:11.343352 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:02:11.343352 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:02:11.343352 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:02:11.343352 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:02:11.343352 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:02:11.343352 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:02:11.343352 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:02:11.343352 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:02:11.343352 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:02:11.344226 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 25 03:02:11.344226 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 25 03:02:11.344226 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 25 03:02:11.344226 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 25 03:02:11.344226 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 25 03:02:11.344226 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 25 03:02:11.344226 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 03:02:11.344226 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:02:11.344226 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 546/1104] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Aug 25 03:02:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:02:11.349540 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d348ed-bcaf-400d-a5b6-78496616a2fe None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:11.445460 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d348ed-bcaf-400d-a5b6-78496616a2fe None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:11.461639 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:11.478577 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e71dd0-11b6-46ea-a59c-4357bf2f7517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:11.479237 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e71dd0-11b6-46ea-a59c-4357bf2f7517 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:11.479519 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e71dd0-11b6-46ea-a59c-4357bf2f7517 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:11.479792 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e71dd0-11b6-46ea-a59c-4357bf2f7517 None admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:11.480479 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e71dd0-11b6-46ea-a59c-4357bf2f7517 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:02:11.480781 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e71dd0-11b6-46ea-a59c-4357bf2f7517 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:11.482193 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e71dd0-11b6-46ea-a59c-4357bf2f7517 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:11.493719 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 559/1105] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Aug 25 03:02:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:02:11.500772 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aee47e3-bebe-497e-921e-a766af1a26af None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:11.567590 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:11.598902 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aee47e3-bebe-497e-921e-a766af1a26af None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:11.632393 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770d1f57-6fdf-4ce8-8fdb-17252f5ddd17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:11.633006 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770d1f57-6fdf-4ce8-8fdb-17252f5ddd17 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:11.633294 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770d1f57-6fdf-4ce8-8fdb-17252f5ddd17 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:11.633572 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770d1f57-6fdf-4ce8-8fdb-17252f5ddd17 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:11.634314 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770d1f57-6fdf-4ce8-8fdb-17252f5ddd17 None admin] RBAC: Adding query filter params (role.id=b8d76e033b4b43ccbe4e580f17be8cc9, scope.project.id=9fd4be43faec4e4191e0e4ec4a4588c3, user.id=0cd171bf4df94f8fab397e42835feeab) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:02:11.634920 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770d1f57-6fdf-4ce8-8fdb-17252f5ddd17 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:11.636044 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770d1f57-6fdf-4ce8-8fdb-17252f5ddd17 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:11.641923 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 547/1106] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Aug 25 03:02:11 2022] GET /identity/v3/role_assignments?user.id=0cd171bf4df94f8fab397e42835feeab&scope.project.id=9fd4be43faec4e4191e0e4ec4a4588c3&role.id=b8d76e033b4b43ccbe4e580f17be8cc9 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:02:13.022705 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d425070-d9ab-43de-98e1-06b40f366c2f None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:13.023230 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d425070-d9ab-43de-98e1-06b40f366c2f None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:13.023322 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d425070-d9ab-43de-98e1-06b40f366c2f None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:13.024605 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 560/1107] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:02:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:02:13.029801 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6dc081-4736-409f-b846-d2b379275faf None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:13.030200 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6dc081-4736-409f-b846-d2b379275faf None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:13.030917 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6dc081-4736-409f-b846-d2b379275faf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:13.049017 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-ec6dc081-4736-409f-b846-d2b379275faf None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:02:13.050205 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-ec6dc081-4736-409f-b846-d2b379275faf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:02:13.113894 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 548/1108] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:02:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 03:02:13.385213 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d425070-d9ab-43de-98e1-06b40f366c2f None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:13.484859 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d425070-d9ab-43de-98e1-06b40f366c2f None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:13.517498 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e220bb2a-a97e-41d4-a0e2-bcbbb1a84554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:13.518120 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e220bb2a-a97e-41d4-a0e2-bcbbb1a84554 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:13.518429 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e220bb2a-a97e-41d4-a0e2-bcbbb1a84554 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:13.518703 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e220bb2a-a97e-41d4-a0e2-bcbbb1a84554 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:13.519793 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e220bb2a-a97e-41d4-a0e2-bcbbb1a84554 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:13.521326 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e220bb2a-a97e-41d4-a0e2-bcbbb1a84554 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:13.522936 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 561/1109] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:02:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:02:13.528681 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6dc081-4736-409f-b846-d2b379275faf None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:13.588269 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:13.626197 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6dc081-4736-409f-b846-d2b379275faf None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:13.659247 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ff9820-afd7-4c61-bac1-5300e1d20e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:13.659858 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ff9820-afd7-4c61-bac1-5300e1d20e63 None admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:13.660135 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ff9820-afd7-4c61-bac1-5300e1d20e63 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:13.660406 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ff9820-afd7-4c61-bac1-5300e1d20e63 None admin] PATH_INFO: `/v3/users` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:13.661082 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ff9820-afd7-4c61-bac1-5300e1d20e63 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:13.662539 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ff9820-afd7-4c61-bac1-5300e1d20e63 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:13.741439 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 549/1110] 10.222.0.58 () {60 vars in 1132 bytes} [Thu Aug 25 03:02:13 2022] POST /identity/v3/users => generated 286 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:02:15.675822 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c48898-63df-427a-b137-9db0087d7099 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:15.676396 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c48898-63df-427a-b137-9db0087d7099 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:15.676396 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c48898-63df-427a-b137-9db0087d7099 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:15.677743 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 562/1111] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:02:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:02:15.683086 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84baff12-b830-478b-8ad3-852bd5a257e9 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:15.683829 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84baff12-b830-478b-8ad3-852bd5a257e9 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:15.683829 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84baff12-b830-478b-8ad3-852bd5a257e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:15.703211 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-84baff12-b830-478b-8ad3-852bd5a257e9 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:02:15.704654 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-84baff12-b830-478b-8ad3-852bd5a257e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:02:15.769881 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 550/1112] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:02:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 03:02:16.038489 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c48898-63df-427a-b137-9db0087d7099 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:16.083923 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:16.136859 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c48898-63df-427a-b137-9db0087d7099 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:16.169132 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48784f2-bde4-43fc-84b2-344aae9ee5c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:16.169752 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48784f2-bde4-43fc-84b2-344aae9ee5c4 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:16.170030 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48784f2-bde4-43fc-84b2-344aae9ee5c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:16.170324 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48784f2-bde4-43fc-84b2-344aae9ee5c4 None admin] PATH_INFO: `/v3/roles/7886cc7e333a42d08085932494a3ea6b` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:16.172240 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48784f2-bde4-43fc-84b2-344aae9ee5c4 None admin] RBAC: Authorizing `identity:get_role(role_id=7886cc7e333a42d08085932494a3ea6b)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:16.173585 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48784f2-bde4-43fc-84b2-344aae9ee5c4 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:16.174921 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 563/1113] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Aug 25 03:02:16 2022] GET /identity/v3/roles/7886cc7e333a42d08085932494a3ea6b => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:02:16.180597 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84baff12-b830-478b-8ad3-852bd5a257e9 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:16.227717 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:16.275443 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84baff12-b830-478b-8ad3-852bd5a257e9 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:16.308901 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d526ff8f-beda-4ab6-a1f8-f02257dbb11b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:16.309459 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d526ff8f-beda-4ab6-a1f8-f02257dbb11b None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:16.309736 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d526ff8f-beda-4ab6-a1f8-f02257dbb11b None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:16.310009 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d526ff8f-beda-4ab6-a1f8-f02257dbb11b None admin] PATH_INFO: `/v3/users/e216f029afae4ab68786c8f1f5bbe1cf` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:16.323053 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d526ff8f-beda-4ab6-a1f8-f02257dbb11b None admin] RBAC: Authorizing `identity:get_user(user_id=e216f029afae4ab68786c8f1f5bbe1cf)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:16.324486 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d526ff8f-beda-4ab6-a1f8-f02257dbb11b None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:16.326048 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 551/1114] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Aug 25 03:02:16 2022] GET /identity/v3/users/e216f029afae4ab68786c8f1f5bbe1cf => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:02:16.332057 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48784f2-bde4-43fc-84b2-344aae9ee5c4 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:16.427718 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48784f2-bde4-43fc-84b2-344aae9ee5c4 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:16.439967 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:16.460709 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d026371-997b-4a79-8330-edcef7f3784d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:16.461307 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d026371-997b-4a79-8330-edcef7f3784d None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:16.461586 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d026371-997b-4a79-8330-edcef7f3784d None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:16.461856 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d026371-997b-4a79-8330-edcef7f3784d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:16.526809 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d026371-997b-4a79-8330-edcef7f3784d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:16.528248 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d026371-997b-4a79-8330-edcef7f3784d None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:16.622218 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 564/1115] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Aug 25 03:02:16 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 25 03:02:16.627998 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d526ff8f-beda-4ab6-a1f8-f02257dbb11b None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:16.723573 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d526ff8f-beda-4ab6-a1f8-f02257dbb11b None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:16.732284 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:16.758917 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb252d2-a0fe-4ad4-b6bc-5b15a3e80d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:16.759549 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb252d2-a0fe-4ad4-b6bc-5b15a3e80d97 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:16.759827 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb252d2-a0fe-4ad4-b6bc-5b15a3e80d97 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:16.760103 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb252d2-a0fe-4ad4-b6bc-5b15a3e80d97 None admin] PATH_INFO: `/v3/projects/4f4073ff732c4d429a6801e8dd46f752` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:16.761212 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb252d2-a0fe-4ad4-b6bc-5b15a3e80d97 None admin] RBAC: Authorizing `identity:get_project(project_id=4f4073ff732c4d429a6801e8dd46f752)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:16.762717 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb252d2-a0fe-4ad4-b6bc-5b15a3e80d97 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:16.764472 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 552/1116] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Aug 25 03:02:16 2022] GET /identity/v3/projects/4f4073ff732c4d429a6801e8dd46f752 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:02:16.771377 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d026371-997b-4a79-8330-edcef7f3784d None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:16.785187 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:16.866891 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d026371-997b-4a79-8330-edcef7f3784d None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:16.900012 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae89917-8acd-4564-adde-b40d0ed2636e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:16.900550 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae89917-8acd-4564-adde-b40d0ed2636e None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:16.900831 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae89917-8acd-4564-adde-b40d0ed2636e None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:16.901105 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae89917-8acd-4564-adde-b40d0ed2636e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:16.901852 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae89917-8acd-4564-adde-b40d0ed2636e None admin] RBAC: Adding query filter params (role.id=7886cc7e333a42d08085932494a3ea6b, scope.project.id=4f4073ff732c4d429a6801e8dd46f752, user.id=e216f029afae4ab68786c8f1f5bbe1cf) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:02:16.902159 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae89917-8acd-4564-adde-b40d0ed2636e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:16.903840 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae89917-8acd-4564-adde-b40d0ed2636e None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:16.909036 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 565/1117] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Aug 25 03:02:16 2022] GET /identity/v3/role_assignments?user.id=e216f029afae4ab68786c8f1f5bbe1cf&scope.project.id=4f4073ff732c4d429a6801e8dd46f752&role.id=7886cc7e333a42d08085932494a3ea6b => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:02:18.778144 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c52bc79-014b-4fc8-b160-e97de49bd6a0 None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:18.778730 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c52bc79-014b-4fc8-b160-e97de49bd6a0 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:18.778838 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c52bc79-014b-4fc8-b160-e97de49bd6a0 None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:18.780099 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 553/1118] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:02:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:02:18.785242 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97006b7d-6e13-49c5-b00a-9f4ddb1888ad None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:18.785659 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97006b7d-6e13-49c5-b00a-9f4ddb1888ad None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:18.785843 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97006b7d-6e13-49c5-b00a-9f4ddb1888ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:18.803780 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-97006b7d-6e13-49c5-b00a-9f4ddb1888ad None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:02:18.804950 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-97006b7d-6e13-49c5-b00a-9f4ddb1888ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:02:18.869950 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 566/1119] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:02:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 03:02:19.139210 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c52bc79-014b-4fc8-b160-e97de49bd6a0 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:19.238431 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c52bc79-014b-4fc8-b160-e97de49bd6a0 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:19.270770 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97cd2ce-533c-462e-8b2d-a01e8592f653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:19.271394 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97cd2ce-533c-462e-8b2d-a01e8592f653 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:19.271677 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97cd2ce-533c-462e-8b2d-a01e8592f653 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:19.271952 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97cd2ce-533c-462e-8b2d-a01e8592f653 None admin] PATH_INFO: `/v3/roles/7886cc7e333a42d08085932494a3ea6b` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:19.273318 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97cd2ce-533c-462e-8b2d-a01e8592f653 None admin] RBAC: Authorizing `identity:get_role(role_id=7886cc7e333a42d08085932494a3ea6b)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:19.274717 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97cd2ce-533c-462e-8b2d-a01e8592f653 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:19.276024 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 554/1120] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Aug 25 03:02:19 2022] GET /identity/v3/roles/7886cc7e333a42d08085932494a3ea6b => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:02:19.282256 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97006b7d-6e13-49c5-b00a-9f4ddb1888ad None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:19.290338 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:19.377440 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97006b7d-6e13-49c5-b00a-9f4ddb1888ad None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:19.410384 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47196625-cbf5-49ef-9fa7-eff02ba455aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:19.411003 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47196625-cbf5-49ef-9fa7-eff02ba455aa None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:19.411285 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47196625-cbf5-49ef-9fa7-eff02ba455aa None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:19.411562 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47196625-cbf5-49ef-9fa7-eff02ba455aa None admin] PATH_INFO: `/v3/users/e216f029afae4ab68786c8f1f5bbe1cf` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:19.412676 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47196625-cbf5-49ef-9fa7-eff02ba455aa None admin] RBAC: Authorizing `identity:get_user(user_id=e216f029afae4ab68786c8f1f5bbe1cf)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:19.414099 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47196625-cbf5-49ef-9fa7-eff02ba455aa None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:19.415682 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 567/1121] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Aug 25 03:02:19 2022] GET /identity/v3/users/e216f029afae4ab68786c8f1f5bbe1cf => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:02:19.421347 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97cd2ce-533c-462e-8b2d-a01e8592f653 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:19.464968 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:19.516468 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97cd2ce-533c-462e-8b2d-a01e8592f653 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:19.548207 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ff26ab-600e-4489-9d54-752a634a8803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:19.548818 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ff26ab-600e-4489-9d54-752a634a8803 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:19.549097 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ff26ab-600e-4489-9d54-752a634a8803 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:19.549368 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ff26ab-600e-4489-9d54-752a634a8803 None admin] PATH_INFO: `/v3/projects/4f4073ff732c4d429a6801e8dd46f752` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:19.550478 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ff26ab-600e-4489-9d54-752a634a8803 None admin] RBAC: Authorizing `identity:get_project(project_id=4f4073ff732c4d429a6801e8dd46f752)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:19.551901 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ff26ab-600e-4489-9d54-752a634a8803 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:19.553488 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 555/1122] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Aug 25 03:02:19 2022] GET /identity/v3/projects/4f4073ff732c4d429a6801e8dd46f752 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:02:19.559294 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47196625-cbf5-49ef-9fa7-eff02ba455aa None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:19.632761 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:19.657197 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47196625-cbf5-49ef-9fa7-eff02ba455aa None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:19.689799 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad848209-5b53-4d1f-b1e3-77627e43f489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:19.690419 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad848209-5b53-4d1f-b1e3-77627e43f489 None admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:19.690702 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad848209-5b53-4d1f-b1e3-77627e43f489 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:19.690980 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad848209-5b53-4d1f-b1e3-77627e43f489 None admin] PATH_INFO: `/v3/projects/4f4073ff732c4d429a6801e8dd46f752/users/e216f029afae4ab68786c8f1f5bbe1cf/roles/7886cc7e333a42d08085932494a3ea6b` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:19.693085 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad848209-5b53-4d1f-b1e3-77627e43f489 None admin] RBAC: Authorizing `identity:create_grant(project_id=4f4073ff732c4d429a6801e8dd46f752, user_id=e216f029afae4ab68786c8f1f5bbe1cf, role_id=7886cc7e333a42d08085932494a3ea6b)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:19.694532 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad848209-5b53-4d1f-b1e3-77627e43f489 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:19.748676 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 568/1123] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Aug 25 03:02:19 2022] PUT /identity/v3/projects/4f4073ff732c4d429a6801e8dd46f752/users/e216f029afae4ab68786c8f1f5bbe1cf/roles/7886cc7e333a42d08085932494a3ea6b => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:02:21.661429 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921f8028-15ac-4d58-bd30-cafecc6492b0 None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:21.661927 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921f8028-15ac-4d58-bd30-cafecc6492b0 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:21.662021 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921f8028-15ac-4d58-bd30-cafecc6492b0 None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:21.663377 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 556/1124] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:02:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:02:21.668645 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebc9af0-5473-415d-b65b-4b70773efbfb None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:21.669056 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebc9af0-5473-415d-b65b-4b70773efbfb None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:21.669200 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebc9af0-5473-415d-b65b-4b70773efbfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:21.689125 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-eebc9af0-5473-415d-b65b-4b70773efbfb None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:02:21.690329 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-eebc9af0-5473-415d-b65b-4b70773efbfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:02:21.754931 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 569/1125] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:02:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 03:02:22.040458 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921f8028-15ac-4d58-bd30-cafecc6492b0 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:22.138386 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921f8028-15ac-4d58-bd30-cafecc6492b0 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:22.150997 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:22.171717 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999f1ce1-cc17-4dfa-9e9c-e008a42d19ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:22.173054 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999f1ce1-cc17-4dfa-9e9c-e008a42d19ca None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:22.173054 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999f1ce1-cc17-4dfa-9e9c-e008a42d19ca None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:22.173303 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999f1ce1-cc17-4dfa-9e9c-e008a42d19ca None admin] PATH_INFO: `/v3/roles/7886cc7e333a42d08085932494a3ea6b` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:22.174731 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999f1ce1-cc17-4dfa-9e9c-e008a42d19ca None admin] RBAC: Authorizing `identity:get_role(role_id=7886cc7e333a42d08085932494a3ea6b)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:22.176105 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999f1ce1-cc17-4dfa-9e9c-e008a42d19ca None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:22.177417 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 557/1126] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Aug 25 03:02:22 2022] GET /identity/v3/roles/7886cc7e333a42d08085932494a3ea6b => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:02:22.183223 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebc9af0-5473-415d-b65b-4b70773efbfb None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:22.247720 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:22.284188 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebc9af0-5473-415d-b65b-4b70773efbfb None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:22.316761 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47eb113-cac6-4f2f-8797-8fa623738b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:22.317329 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47eb113-cac6-4f2f-8797-8fa623738b15 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:22.317608 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47eb113-cac6-4f2f-8797-8fa623738b15 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:22.317879 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47eb113-cac6-4f2f-8797-8fa623738b15 None admin] PATH_INFO: `/v3/users/e216f029afae4ab68786c8f1f5bbe1cf` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:22.318989 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47eb113-cac6-4f2f-8797-8fa623738b15 None admin] RBAC: Authorizing `identity:get_user(user_id=e216f029afae4ab68786c8f1f5bbe1cf)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:22.320435 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47eb113-cac6-4f2f-8797-8fa623738b15 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:22.321993 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 570/1127] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Aug 25 03:02:22 2022] GET /identity/v3/users/e216f029afae4ab68786c8f1f5bbe1cf => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:02:22.327867 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999f1ce1-cc17-4dfa-9e9c-e008a42d19ca None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:22.423852 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999f1ce1-cc17-4dfa-9e9c-e008a42d19ca None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:22.455531 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ed6357-6e63-48a5-96dc-971d82114cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:22.456119 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ed6357-6e63-48a5-96dc-971d82114cf1 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:22.456397 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ed6357-6e63-48a5-96dc-971d82114cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:22.456670 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ed6357-6e63-48a5-96dc-971d82114cf1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:22.502116 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:22.520497 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ed6357-6e63-48a5-96dc-971d82114cf1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:22.521952 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ed6357-6e63-48a5-96dc-971d82114cf1 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:22.615646 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 558/1128] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Aug 25 03:02:22 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 25 03:02:22.621488 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47eb113-cac6-4f2f-8797-8fa623738b15 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:22.721490 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47eb113-cac6-4f2f-8797-8fa623738b15 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:22.748734 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:22.753859 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5a15d2-f0a4-4ea0-821b-156f98a0b689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:22.754518 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5a15d2-f0a4-4ea0-821b-156f98a0b689 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:22.754800 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5a15d2-f0a4-4ea0-821b-156f98a0b689 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:22.755073 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5a15d2-f0a4-4ea0-821b-156f98a0b689 None admin] PATH_INFO: `/v3/projects/4f4073ff732c4d429a6801e8dd46f752` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:22.756165 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5a15d2-f0a4-4ea0-821b-156f98a0b689 None admin] RBAC: Authorizing `identity:get_project(project_id=4f4073ff732c4d429a6801e8dd46f752)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:22.757591 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5a15d2-f0a4-4ea0-821b-156f98a0b689 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:22.759253 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 571/1129] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Aug 25 03:02:22 2022] GET /identity/v3/projects/4f4073ff732c4d429a6801e8dd46f752 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:02:22.766052 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ed6357-6e63-48a5-96dc-971d82114cf1 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:22.841365 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:22.859038 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ed6357-6e63-48a5-96dc-971d82114cf1 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:22.892200 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca97da4b-c69f-4c81-b553-f983f50da850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:22.892726 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca97da4b-c69f-4c81-b553-f983f50da850 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:22.893002 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca97da4b-c69f-4c81-b553-f983f50da850 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:22.893271 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca97da4b-c69f-4c81-b553-f983f50da850 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:22.893982 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca97da4b-c69f-4c81-b553-f983f50da850 None admin] RBAC: Adding query filter params (role.id=7886cc7e333a42d08085932494a3ea6b, scope.project.id=4f4073ff732c4d429a6801e8dd46f752, user.id=e216f029afae4ab68786c8f1f5bbe1cf) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:02:22.894309 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca97da4b-c69f-4c81-b553-f983f50da850 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:22.895717 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca97da4b-c69f-4c81-b553-f983f50da850 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:22.901230 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 559/1130] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Aug 25 03:02:22 2022] GET /identity/v3/role_assignments?user.id=e216f029afae4ab68786c8f1f5bbe1cf&scope.project.id=4f4073ff732c4d429a6801e8dd46f752&role.id=7886cc7e333a42d08085932494a3ea6b => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:02:24.798110 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1e007c-2f51-4ae4-a8af-a17a91a446f4 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:24.798654 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1e007c-2f51-4ae4-a8af-a17a91a446f4 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:24.798745 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1e007c-2f51-4ae4-a8af-a17a91a446f4 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:24.800025 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 572/1131] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:02:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:02:24.805257 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effda9ce-1ef7-4cd1-81dc-a8cf8be16a5b None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:24.805701 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effda9ce-1ef7-4cd1-81dc-a8cf8be16a5b None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:24.805796 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effda9ce-1ef7-4cd1-81dc-a8cf8be16a5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:24.824009 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-effda9ce-1ef7-4cd1-81dc-a8cf8be16a5b None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:02:24.825199 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-effda9ce-1ef7-4cd1-81dc-a8cf8be16a5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:02:24.889820 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 560/1132] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:02:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 03:02:25.157261 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1e007c-2f51-4ae4-a8af-a17a91a446f4 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:25.257417 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1e007c-2f51-4ae4-a8af-a17a91a446f4 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:25.298927 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb93d460-8294-4b63-bb9c-ddf51d2a3481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:25.299559 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb93d460-8294-4b63-bb9c-ddf51d2a3481 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:25.299839 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb93d460-8294-4b63-bb9c-ddf51d2a3481 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:25.300111 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb93d460-8294-4b63-bb9c-ddf51d2a3481 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:25.301223 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb93d460-8294-4b63-bb9c-ddf51d2a3481 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:25.302665 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb93d460-8294-4b63-bb9c-ddf51d2a3481 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:25.308057 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 573/1133] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:02:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:02:25.314516 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effda9ce-1ef7-4cd1-81dc-a8cf8be16a5b None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:25.386411 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:25.410743 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effda9ce-1ef7-4cd1-81dc-a8cf8be16a5b None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:25.443433 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28b16bd-8226-4f74-8de5-59fee0d3e2cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:25.444030 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28b16bd-8226-4f74-8de5-59fee0d3e2cf None admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:25.444310 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28b16bd-8226-4f74-8de5-59fee0d3e2cf None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:25.444580 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28b16bd-8226-4f74-8de5-59fee0d3e2cf None admin] PATH_INFO: `/v3/projects` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:25.445216 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28b16bd-8226-4f74-8de5-59fee0d3e2cf None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:25.446674 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28b16bd-8226-4f74-8de5-59fee0d3e2cf None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:25.539173 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 561/1134] 10.222.0.58 () {60 vars in 1141 bytes} [Thu Aug 25 03:02:25 2022] POST /identity/v3/projects => generated 311 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:02:27.479695 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705600e8-b6d5-43ef-9075-1450f78e833d None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:27.480223 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705600e8-b6d5-43ef-9075-1450f78e833d None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:27.480223 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705600e8-b6d5-43ef-9075-1450f78e833d None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:27.481569 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 574/1135] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:02:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:02:27.486908 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d22df95-74b6-4da2-9051-36a370819c61 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:27.487295 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d22df95-74b6-4da2-9051-36a370819c61 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:27.487460 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d22df95-74b6-4da2-9051-36a370819c61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:27.505414 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-1d22df95-74b6-4da2-9051-36a370819c61 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:02:27.506660 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-1d22df95-74b6-4da2-9051-36a370819c61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:02:27.570594 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 562/1136] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:02:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 03:02:27.840398 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705600e8-b6d5-43ef-9075-1450f78e833d None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:27.907097 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:27.941255 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705600e8-b6d5-43ef-9075-1450f78e833d None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:27.973274 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c3c5f8-71f3-412e-a33b-a95cae6b1f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:27.973896 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c3c5f8-71f3-412e-a33b-a95cae6b1f00 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:27.974173 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c3c5f8-71f3-412e-a33b-a95cae6b1f00 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:27.974564 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c3c5f8-71f3-412e-a33b-a95cae6b1f00 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:27.979856 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c3c5f8-71f3-412e-a33b-a95cae6b1f00 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:27.979856 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c3c5f8-71f3-412e-a33b-a95cae6b1f00 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:27.979856 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 575/1137] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:02:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:02:27.985010 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d22df95-74b6-4da2-9051-36a370819c61 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:28.039763 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:28.081506 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d22df95-74b6-4da2-9051-36a370819c61 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:28.114955 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b03db01-8056-4477-8bf0-eb20837fa845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:28.115494 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03db01-8056-4477-8bf0-eb20837fa845 None admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:28.115775 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03db01-8056-4477-8bf0-eb20837fa845 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:28.116051 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03db01-8056-4477-8bf0-eb20837fa845 None admin] PATH_INFO: `/v3/users` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:28.116689 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b03db01-8056-4477-8bf0-eb20837fa845 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:28.118116 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b03db01-8056-4477-8bf0-eb20837fa845 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:28.272934 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 563/1138] 10.222.0.58 () {60 vars in 1132 bytes} [Thu Aug 25 03:02:27 2022] POST /identity/v3/users => generated 286 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:02:30.211587 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c185a32-ea48-460a-8c40-de16423763ef None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:30.212144 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c185a32-ea48-460a-8c40-de16423763ef None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:30.212144 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c185a32-ea48-460a-8c40-de16423763ef None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:30.213448 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 576/1139] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:02:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:02:30.219208 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cabd3d-a2ac-4772-8494-1fa6a1e21e9c None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:30.219627 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cabd3d-a2ac-4772-8494-1fa6a1e21e9c None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:30.219736 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cabd3d-a2ac-4772-8494-1fa6a1e21e9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:30.237225 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-b6cabd3d-a2ac-4772-8494-1fa6a1e21e9c None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:02:30.238424 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-b6cabd3d-a2ac-4772-8494-1fa6a1e21e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:02:30.301936 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 564/1140] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:02:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 03:02:30.569333 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c185a32-ea48-460a-8c40-de16423763ef None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:30.667912 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c185a32-ea48-460a-8c40-de16423763ef None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:30.700465 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2a31db-2d31-426f-880f-f77fefce7be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:30.701075 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2a31db-2d31-426f-880f-f77fefce7be4 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:30.701350 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2a31db-2d31-426f-880f-f77fefce7be4 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:30.701616 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2a31db-2d31-426f-880f-f77fefce7be4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:30.704495 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:30.712014 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2a31db-2d31-426f-880f-f77fefce7be4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:02:30.718132 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-da2a31db-2d31-426f-880f-f77fefce7be4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 03:02:30.718132 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:02:30.718132 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:02:30.718132 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:02:30.718132 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:02:30.718132 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:02:30.718132 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:02:30.718132 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:02:30.718132 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:02:30.718132 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:02:30.718132 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:02:30.718132 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:02:30.718132 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:02:30.718132 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:02:30.718132 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:02:30.718132 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:02:30.719340 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:02:30.719340 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:02:30.719340 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:02:30.719340 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:02:30.719340 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:02:30.719340 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:02:30.719340 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:02:30.719340 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:02:30.719340 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:02:30.719340 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:02:30.719340 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:02:30.719340 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:02:30.719340 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:02:30.719340 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:02:30.719340 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:02:30.719340 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:02:30.720472 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:02:30.720472 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:02:30.720472 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:02:30.720472 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:02:30.720472 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:02:30.720472 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:02:30.720472 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:02:30.720472 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:02:30.720472 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:02:30.720472 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:02:30.720472 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 03:02:30.720472 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:02:30.720472 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 577/1141] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Aug 25 03:02:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:02:30.723774 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cabd3d-a2ac-4772-8494-1fa6a1e21e9c None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:30.766637 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:30.820553 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cabd3d-a2ac-4772-8494-1fa6a1e21e9c None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:30.853319 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cede60b4-4b2e-44bb-938d-ee245d3176ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:30.853943 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cede60b4-4b2e-44bb-938d-ee245d3176ef None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:30.854224 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cede60b4-4b2e-44bb-938d-ee245d3176ef None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:30.854528 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cede60b4-4b2e-44bb-938d-ee245d3176ef None admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:30.855189 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cede60b4-4b2e-44bb-938d-ee245d3176ef None admin] RBAC: Adding query filter params (name=admin) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:02:30.855493 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cede60b4-4b2e-44bb-938d-ee245d3176ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:30.856854 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cede60b4-4b2e-44bb-938d-ee245d3176ef None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:30.865022 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 565/1142] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Aug 25 03:02:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:02:30.870846 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2a31db-2d31-426f-880f-f77fefce7be4 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:30.966317 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2a31db-2d31-426f-880f-f77fefce7be4 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:30.998661 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d520d9ef-efc7-4d03-a49b-6b97776d6dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:30.999310 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d520d9ef-efc7-4d03-a49b-6b97776d6dc1 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:31.000198 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d520d9ef-efc7-4d03-a49b-6b97776d6dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:31.000466 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d520d9ef-efc7-4d03-a49b-6b97776d6dc1 None admin] PATH_INFO: `/v3/users/0811e9bdb2f54d02a21163b2c48d07dd` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:31.013102 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d520d9ef-efc7-4d03-a49b-6b97776d6dc1 None admin] RBAC: Authorizing `identity:get_user(user_id=0811e9bdb2f54d02a21163b2c48d07dd)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:31.014541 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d520d9ef-efc7-4d03-a49b-6b97776d6dc1 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:31.016240 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 578/1143] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Aug 25 03:02:30 2022] GET /identity/v3/users/0811e9bdb2f54d02a21163b2c48d07dd => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:02:31.022195 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cede60b4-4b2e-44bb-938d-ee245d3176ef None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:31.123902 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cede60b4-4b2e-44bb-938d-ee245d3176ef None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:31.126131 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:31.156441 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df72f145-c014-4b60-bf76-c7264dc5f0d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:31.157040 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df72f145-c014-4b60-bf76-c7264dc5f0d9 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:31.157321 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df72f145-c014-4b60-bf76-c7264dc5f0d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:31.157595 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df72f145-c014-4b60-bf76-c7264dc5f0d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:31.222755 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df72f145-c014-4b60-bf76-c7264dc5f0d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:31.224188 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df72f145-c014-4b60-bf76-c7264dc5f0d9 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:31.316626 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 566/1144] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Aug 25 03:02:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 25 03:02:31.322506 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d520d9ef-efc7-4d03-a49b-6b97776d6dc1 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:31.376353 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:31.419615 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d520d9ef-efc7-4d03-a49b-6b97776d6dc1 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:31.450653 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c606f6a1-fb6c-4330-9da1-c3b23e777d27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:31.451263 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c606f6a1-fb6c-4330-9da1-c3b23e777d27 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:31.451539 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c606f6a1-fb6c-4330-9da1-c3b23e777d27 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:31.451807 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c606f6a1-fb6c-4330-9da1-c3b23e777d27 None admin] PATH_INFO: `/v3/projects/0900aee3acf647f594a7bab0868ed2ad` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:31.453378 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c606f6a1-fb6c-4330-9da1-c3b23e777d27 None admin] RBAC: Authorizing `identity:get_project(project_id=0900aee3acf647f594a7bab0868ed2ad)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:31.455079 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c606f6a1-fb6c-4330-9da1-c3b23e777d27 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:31.456666 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 579/1145] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Aug 25 03:02:31 2022] GET /identity/v3/projects/0900aee3acf647f594a7bab0868ed2ad => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:02:31.462643 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df72f145-c014-4b60-bf76-c7264dc5f0d9 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:31.465661 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:31.558388 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df72f145-c014-4b60-bf76-c7264dc5f0d9 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:31.591356 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e915fc-d8d6-4b7e-8bf0-cfbea4599ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:31.591961 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e915fc-d8d6-4b7e-8bf0-cfbea4599ed5 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:31.592241 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e915fc-d8d6-4b7e-8bf0-cfbea4599ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:31.592516 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e915fc-d8d6-4b7e-8bf0-cfbea4599ed5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:31.593859 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e915fc-d8d6-4b7e-8bf0-cfbea4599ed5 None admin] RBAC: Adding query filter params (role.id=62f19ea272b447ccaab1576cd9bf4fd9, scope.project.id=0900aee3acf647f594a7bab0868ed2ad, user.id=0811e9bdb2f54d02a21163b2c48d07dd) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:02:31.594167 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e915fc-d8d6-4b7e-8bf0-cfbea4599ed5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:31.595637 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e915fc-d8d6-4b7e-8bf0-cfbea4599ed5 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:31.600850 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 567/1146] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Aug 25 03:02:31 2022] GET /identity/v3/role_assignments?user.id=0811e9bdb2f54d02a21163b2c48d07dd&scope.project.id=0900aee3acf647f594a7bab0868ed2ad&role.id=62f19ea272b447ccaab1576cd9bf4fd9 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:02:33.463034 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b981b5-75c7-44c5-8f0d-02a07f6f4137 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:33.463601 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b981b5-75c7-44c5-8f0d-02a07f6f4137 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:33.463601 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b981b5-75c7-44c5-8f0d-02a07f6f4137 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:33.464901 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 580/1147] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:02:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:02:33.470211 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eceba4-3644-44a4-ab00-df053329fb3b None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:33.470678 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eceba4-3644-44a4-ab00-df053329fb3b None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:33.470786 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eceba4-3644-44a4-ab00-df053329fb3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:33.488334 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-48eceba4-3644-44a4-ab00-df053329fb3b None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:02:33.489517 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-48eceba4-3644-44a4-ab00-df053329fb3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:02:33.547918 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:33.553128 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 568/1148] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:02:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 03:02:33.821434 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b981b5-75c7-44c5-8f0d-02a07f6f4137 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:33.925351 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b981b5-75c7-44c5-8f0d-02a07f6f4137 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:33.946959 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:33.958809 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ae5356-abc5-4126-bcda-58271be5b5df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:33.959433 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ae5356-abc5-4126-bcda-58271be5b5df None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:33.959716 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ae5356-abc5-4126-bcda-58271be5b5df None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:33.959990 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ae5356-abc5-4126-bcda-58271be5b5df None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:33.970256 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ae5356-abc5-4126-bcda-58271be5b5df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:02:33.974924 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-38ae5356-abc5-4126-bcda-58271be5b5df None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 03:02:33.974924 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:02:33.974924 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:02:33.974924 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:02:33.974924 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:02:33.974924 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:02:33.974924 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:02:33.974924 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:02:33.974924 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:02:33.974924 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:02:33.974924 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:02:33.974924 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:02:33.974924 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:02:33.974924 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:02:33.974924 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:02:33.974924 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:02:33.975863 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:02:33.975863 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:02:33.975863 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:02:33.975863 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:02:33.975863 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:02:33.975863 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:02:33.975863 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:02:33.975863 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:02:33.975863 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:02:33.975863 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:02:33.975863 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:02:33.975863 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:02:33.975863 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:02:33.975863 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:02:33.975863 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:02:33.975863 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:02:33.976703 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:02:33.976703 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:02:33.976703 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:02:33.976703 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:02:33.976703 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:02:33.976703 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:02:33.976703 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:02:33.976703 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:02:33.976703 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:02:33.976703 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:02:33.976703 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 03:02:33.976703 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:02:33.976703 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 581/1149] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Aug 25 03:02:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:02:33.982167 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eceba4-3644-44a4-ab00-df053329fb3b None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:34.075901 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eceba4-3644-44a4-ab00-df053329fb3b None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:34.108545 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816ee95c-95fe-4ace-8f25-eabd54dc7545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:34.109137 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816ee95c-95fe-4ace-8f25-eabd54dc7545 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:34.109413 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816ee95c-95fe-4ace-8f25-eabd54dc7545 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:34.109679 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816ee95c-95fe-4ace-8f25-eabd54dc7545 None admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:34.110353 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816ee95c-95fe-4ace-8f25-eabd54dc7545 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:02:34.110655 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816ee95c-95fe-4ace-8f25-eabd54dc7545 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:34.111998 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816ee95c-95fe-4ace-8f25-eabd54dc7545 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:34.119653 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 569/1150] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Aug 25 03:02:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:02:34.125753 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ae5356-abc5-4126-bcda-58271be5b5df None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:34.244936 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ae5356-abc5-4126-bcda-58271be5b5df None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:34.278004 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62caf19-fc1b-4d5c-9316-7abd35396064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:34.278644 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62caf19-fc1b-4d5c-9316-7abd35396064 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:34.278921 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62caf19-fc1b-4d5c-9316-7abd35396064 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:34.279188 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62caf19-fc1b-4d5c-9316-7abd35396064 None admin] PATH_INFO: `/v3/users/0811e9bdb2f54d02a21163b2c48d07dd` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:34.280274 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62caf19-fc1b-4d5c-9316-7abd35396064 None admin] RBAC: Authorizing `identity:get_user(user_id=0811e9bdb2f54d02a21163b2c48d07dd)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:34.281741 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62caf19-fc1b-4d5c-9316-7abd35396064 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:34.283396 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 582/1151] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Aug 25 03:02:34 2022] GET /identity/v3/users/0811e9bdb2f54d02a21163b2c48d07dd => generated 286 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:02:34.289184 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816ee95c-95fe-4ace-8f25-eabd54dc7545 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:34.346917 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:34.385602 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816ee95c-95fe-4ace-8f25-eabd54dc7545 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:34.417652 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fc3a0e-78fe-4fae-9a3d-d92144506e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:34.418270 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fc3a0e-78fe-4fae-9a3d-d92144506e21 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:34.418579 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fc3a0e-78fe-4fae-9a3d-d92144506e21 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:34.418853 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fc3a0e-78fe-4fae-9a3d-d92144506e21 None admin] PATH_INFO: `/v3/projects/0900aee3acf647f594a7bab0868ed2ad` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:34.419951 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fc3a0e-78fe-4fae-9a3d-d92144506e21 None admin] RBAC: Authorizing `identity:get_project(project_id=0900aee3acf647f594a7bab0868ed2ad)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:34.421895 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fc3a0e-78fe-4fae-9a3d-d92144506e21 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:34.423531 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 570/1152] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Aug 25 03:02:34 2022] GET /identity/v3/projects/0900aee3acf647f594a7bab0868ed2ad => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:02:34.429311 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62caf19-fc1b-4d5c-9316-7abd35396064 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:34.476916 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:34.525187 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62caf19-fc1b-4d5c-9316-7abd35396064 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:34.557506 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050dfb7b-7818-4d1a-9e26-71db79d49209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:34.558150 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050dfb7b-7818-4d1a-9e26-71db79d49209 None admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:34.558481 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050dfb7b-7818-4d1a-9e26-71db79d49209 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:34.558757 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050dfb7b-7818-4d1a-9e26-71db79d49209 None admin] PATH_INFO: `/v3/projects/0900aee3acf647f594a7bab0868ed2ad/users/0811e9bdb2f54d02a21163b2c48d07dd/roles/62f19ea272b447ccaab1576cd9bf4fd9` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:34.560615 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050dfb7b-7818-4d1a-9e26-71db79d49209 None admin] RBAC: Authorizing `identity:create_grant(project_id=0900aee3acf647f594a7bab0868ed2ad, user_id=0811e9bdb2f54d02a21163b2c48d07dd, role_id=62f19ea272b447ccaab1576cd9bf4fd9)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:34.562645 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050dfb7b-7818-4d1a-9e26-71db79d49209 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:34.634925 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 583/1153] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Aug 25 03:02:34 2022] PUT /identity/v3/projects/0900aee3acf647f594a7bab0868ed2ad/users/0811e9bdb2f54d02a21163b2c48d07dd/roles/62f19ea272b447ccaab1576cd9bf4fd9 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:02:36.508418 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d438296-ddac-46da-af5e-6017dfd3ba03 None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:36.508947 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d438296-ddac-46da-af5e-6017dfd3ba03 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:36.508947 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d438296-ddac-46da-af5e-6017dfd3ba03 None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:36.510309 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 571/1154] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:02:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:02:36.516259 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb92385-0a52-423c-8728-2cfc8d4bcb4c None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:36.516659 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb92385-0a52-423c-8728-2cfc8d4bcb4c None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:36.516802 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb92385-0a52-423c-8728-2cfc8d4bcb4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:36.534373 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-adb92385-0a52-423c-8728-2cfc8d4bcb4c None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:02:36.535551 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-adb92385-0a52-423c-8728-2cfc8d4bcb4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:02:36.600100 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 584/1155] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:02:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 03:02:36.865845 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d438296-ddac-46da-af5e-6017dfd3ba03 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:36.973888 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d438296-ddac-46da-af5e-6017dfd3ba03 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:36.999401 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:37.007701 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f076ba3-141f-4434-bf22-6cbfb82483ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:37.008335 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f076ba3-141f-4434-bf22-6cbfb82483ae None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:37.008615 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f076ba3-141f-4434-bf22-6cbfb82483ae None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:37.008888 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f076ba3-141f-4434-bf22-6cbfb82483ae None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:37.018833 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f076ba3-141f-4434-bf22-6cbfb82483ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:02:37.023652 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-6f076ba3-141f-4434-bf22-6cbfb82483ae None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 03:02:37.023652 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:02:37.023652 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:02:37.023652 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:02:37.023652 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:02:37.023652 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:02:37.023652 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:02:37.023652 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:02:37.023652 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:02:37.023652 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:02:37.023652 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:02:37.023652 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:02:37.023652 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:02:37.023652 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:02:37.023652 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:02:37.023652 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:02:37.024776 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:02:37.024776 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:02:37.024776 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:02:37.024776 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:02:37.024776 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:02:37.024776 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:02:37.024776 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:02:37.024776 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:02:37.024776 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:02:37.024776 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:02:37.024776 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:02:37.024776 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:02:37.024776 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:02:37.024776 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:02:37.024776 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:02:37.024776 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:02:37.025782 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:02:37.025782 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:02:37.025782 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:02:37.025782 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:02:37.025782 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:02:37.025782 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:02:37.025782 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:02:37.025782 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:02:37.025782 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:02:37.025782 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:02:37.025782 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 03:02:37.025782 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:02:37.025782 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 572/1156] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Aug 25 03:02:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:02:37.031384 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb92385-0a52-423c-8728-2cfc8d4bcb4c None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:37.071043 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:37.127420 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb92385-0a52-423c-8728-2cfc8d4bcb4c None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:37.159882 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f42512-689a-43b6-8e67-8e82fcff7687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:37.160636 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f42512-689a-43b6-8e67-8e82fcff7687 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:37.161457 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f42512-689a-43b6-8e67-8e82fcff7687 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:37.161737 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f42512-689a-43b6-8e67-8e82fcff7687 None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:37.162438 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f42512-689a-43b6-8e67-8e82fcff7687 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:02:37.162747 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f42512-689a-43b6-8e67-8e82fcff7687 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:37.164109 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f42512-689a-43b6-8e67-8e82fcff7687 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:37.172429 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 585/1157] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Aug 25 03:02:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:02:37.178228 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f076ba3-141f-4434-bf22-6cbfb82483ae None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:37.279797 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f076ba3-141f-4434-bf22-6cbfb82483ae None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:37.313075 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3cfffd-2ac5-46e0-a83c-6ae4d84abd8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:37.313694 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3cfffd-2ac5-46e0-a83c-6ae4d84abd8e None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:37.313970 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3cfffd-2ac5-46e0-a83c-6ae4d84abd8e None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:37.314240 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3cfffd-2ac5-46e0-a83c-6ae4d84abd8e None admin] PATH_INFO: `/v3/users/0811e9bdb2f54d02a21163b2c48d07dd` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:37.315377 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3cfffd-2ac5-46e0-a83c-6ae4d84abd8e None admin] RBAC: Authorizing `identity:get_user(user_id=0811e9bdb2f54d02a21163b2c48d07dd)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:37.316801 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3cfffd-2ac5-46e0-a83c-6ae4d84abd8e None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:37.318374 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 573/1158] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Aug 25 03:02:37 2022] GET /identity/v3/users/0811e9bdb2f54d02a21163b2c48d07dd => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:02:37.324477 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f42512-689a-43b6-8e67-8e82fcff7687 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:37.421511 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f42512-689a-43b6-8e67-8e82fcff7687 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:37.423046 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:37.454887 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498f3a61-9892-41a2-a8c5-e9f88d6ac8ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:37.455479 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498f3a61-9892-41a2-a8c5-e9f88d6ac8ee None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:37.455760 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498f3a61-9892-41a2-a8c5-e9f88d6ac8ee None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:37.456034 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498f3a61-9892-41a2-a8c5-e9f88d6ac8ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:37.519432 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498f3a61-9892-41a2-a8c5-e9f88d6ac8ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:37.520851 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498f3a61-9892-41a2-a8c5-e9f88d6ac8ee None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:37.619871 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 586/1159] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Aug 25 03:02:37 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 25 03:02:37.625747 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3cfffd-2ac5-46e0-a83c-6ae4d84abd8e None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:37.679921 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:37.722857 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3cfffd-2ac5-46e0-a83c-6ae4d84abd8e None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:37.754774 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c196694d-4454-4912-8dd4-4ad60287a233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:37.755382 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c196694d-4454-4912-8dd4-4ad60287a233 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:37.755660 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c196694d-4454-4912-8dd4-4ad60287a233 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:37.755930 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c196694d-4454-4912-8dd4-4ad60287a233 None admin] PATH_INFO: `/v3/projects/0900aee3acf647f594a7bab0868ed2ad` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:37.757015 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c196694d-4454-4912-8dd4-4ad60287a233 None admin] RBAC: Authorizing `identity:get_project(project_id=0900aee3acf647f594a7bab0868ed2ad)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:37.758457 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c196694d-4454-4912-8dd4-4ad60287a233 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:37.760050 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 574/1160] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Aug 25 03:02:37 2022] GET /identity/v3/projects/0900aee3acf647f594a7bab0868ed2ad => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:02:37.766994 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498f3a61-9892-41a2-a8c5-e9f88d6ac8ee None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:37.769194 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:37.863739 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498f3a61-9892-41a2-a8c5-e9f88d6ac8ee None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:37.896381 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ae9146-ef58-4588-a74d-4eebc5a8e577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:37.896966 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ae9146-ef58-4588-a74d-4eebc5a8e577 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:37.897241 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ae9146-ef58-4588-a74d-4eebc5a8e577 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:37.897508 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ae9146-ef58-4588-a74d-4eebc5a8e577 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:37.898215 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ae9146-ef58-4588-a74d-4eebc5a8e577 None admin] RBAC: Adding query filter params (role.id=62f19ea272b447ccaab1576cd9bf4fd9, scope.project.id=0900aee3acf647f594a7bab0868ed2ad, user.id=0811e9bdb2f54d02a21163b2c48d07dd) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:02:37.898543 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ae9146-ef58-4588-a74d-4eebc5a8e577 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:37.899960 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ae9146-ef58-4588-a74d-4eebc5a8e577 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:37.905515 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 587/1161] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Aug 25 03:02:37 2022] GET /identity/v3/role_assignments?user.id=0811e9bdb2f54d02a21163b2c48d07dd&scope.project.id=0900aee3acf647f594a7bab0868ed2ad&role.id=62f19ea272b447ccaab1576cd9bf4fd9 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:02:39.826975 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435e3927-79eb-4cf3-bb82-138dd1c39316 None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:39.827967 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435e3927-79eb-4cf3-bb82-138dd1c39316 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:39.828062 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435e3927-79eb-4cf3-bb82-138dd1c39316 None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:39.829582 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 575/1162] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:02:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:02:39.834711 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a971f696-c69d-4c6f-b81a-28c970ecdc88 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:39.835090 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a971f696-c69d-4c6f-b81a-28c970ecdc88 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:39.835274 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a971f696-c69d-4c6f-b81a-28c970ecdc88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:39.853376 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-a971f696-c69d-4c6f-b81a-28c970ecdc88 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:02:39.854590 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-a971f696-c69d-4c6f-b81a-28c970ecdc88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:02:39.911945 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:39.919120 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 588/1163] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:02:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 03:02:40.190565 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435e3927-79eb-4cf3-bb82-138dd1c39316 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:40.287658 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435e3927-79eb-4cf3-bb82-138dd1c39316 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:40.309486 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:40.320060 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f70234e-fb7e-4491-adc0-45bfe1c6aed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:40.320657 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f70234e-fb7e-4491-adc0-45bfe1c6aed7 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:40.323547 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f70234e-fb7e-4491-adc0-45bfe1c6aed7 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:40.323547 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f70234e-fb7e-4491-adc0-45bfe1c6aed7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:40.386514 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f70234e-fb7e-4491-adc0-45bfe1c6aed7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:40.387958 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f70234e-fb7e-4491-adc0-45bfe1c6aed7 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:40.483329 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 576/1164] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Aug 25 03:02:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 25 03:02:40.489737 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a971f696-c69d-4c6f-b81a-28c970ecdc88 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:40.585074 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a971f696-c69d-4c6f-b81a-28c970ecdc88 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:40.617809 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a612987-9fc3-48fc-8d7e-9e78062f684d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:40.618457 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a612987-9fc3-48fc-8d7e-9e78062f684d None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:40.618741 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a612987-9fc3-48fc-8d7e-9e78062f684d None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:40.619015 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a612987-9fc3-48fc-8d7e-9e78062f684d None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:40.626007 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a612987-9fc3-48fc-8d7e-9e78062f684d None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:40.627473 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a612987-9fc3-48fc-8d7e-9e78062f684d None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:40.638987 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-4a612987-9fc3-48fc-8d7e-9e78062f684d None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 25 03:02:40.638987 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:02:40.638987 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 25 03:02:40.638987 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 25 03:02:40.638987 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 25 03:02:40.638987 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 25 03:02:40.638987 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 25 03:02:40.638987 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 25 03:02:40.638987 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 25 03:02:40.638987 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:02:40.638987 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 25 03:02:40.638987 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:02:40.638987 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:02:40.638987 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:02:40.638987 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:02:40.638987 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:02:40.638987 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:02:40.640233 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:02:40.640233 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:02:40.640233 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:02:40.640233 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:02:40.640233 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:02:40.640233 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:02:40.640233 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 03:02:40.640233 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 03:02:40.640233 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 03:02:40.640233 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 03:02:40.640233 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:02:40.640233 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:02:40.640233 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:02:40.640233 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:02:40.640233 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:02:40.640233 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:02:40.641127 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:02:40.641127 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:02:40.641127 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:02:40.641127 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:02:40.641127 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:02:40.641127 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:02:40.641127 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:02:40.641127 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:02:40.641127 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:02:40.641127 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:02:40.641127 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 25 03:02:40.641127 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 03:02:40.641127 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 25 03:02:40.641127 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:02:40.641127 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 589/1165] 10.222.0.58 () {56 vars in 1117 bytes} [Thu Aug 25 03:02:40 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:02:40.647178 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f70234e-fb7e-4491-adc0-45bfe1c6aed7 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:40.674411 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:40.744917 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f70234e-fb7e-4491-adc0-45bfe1c6aed7 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:40.776864 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573e15dc-bb12-4152-bf3d-40ff8065a06b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:40.777455 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573e15dc-bb12-4152-bf3d-40ff8065a06b None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:40.777731 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573e15dc-bb12-4152-bf3d-40ff8065a06b None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:40.778000 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573e15dc-bb12-4152-bf3d-40ff8065a06b None admin] PATH_INFO: `/v3/domains` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:40.778655 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573e15dc-bb12-4152-bf3d-40ff8065a06b None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:02:40.778951 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573e15dc-bb12-4152-bf3d-40ff8065a06b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:40.780322 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573e15dc-bb12-4152-bf3d-40ff8065a06b None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:40.787605 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 577/1166] 10.222.0.58 () {56 vars in 1115 bytes} [Thu Aug 25 03:02:40 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:02:42.680083 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90ce122-7810-4bda-b340-ca92ed0690c2 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:42.680521 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90ce122-7810-4bda-b340-ca92ed0690c2 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:42.680636 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90ce122-7810-4bda-b340-ca92ed0690c2 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:42.682001 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 590/1167] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:02:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:02:42.688166 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e90710-f67b-4584-9b73-f4aa6c1ecffd None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:42.688566 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e90710-f67b-4584-9b73-f4aa6c1ecffd None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:42.688713 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e90710-f67b-4584-9b73-f4aa6c1ecffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:42.707194 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-f9e90710-f67b-4584-9b73-f4aa6c1ecffd None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:02:42.708369 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-f9e90710-f67b-4584-9b73-f4aa6c1ecffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:02:42.771806 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 578/1168] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:02:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 03:02:43.042293 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90ce122-7810-4bda-b340-ca92ed0690c2 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:43.098102 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:43.140440 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90ce122-7810-4bda-b340-ca92ed0690c2 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:43.172567 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbe9c0a-396b-4142-ad9f-d2e65518339f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:43.173153 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbe9c0a-396b-4142-ad9f-d2e65518339f None admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:43.173428 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbe9c0a-396b-4142-ad9f-d2e65518339f None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:43.173697 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbe9c0a-396b-4142-ad9f-d2e65518339f None admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:43.174383 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbe9c0a-396b-4142-ad9f-d2e65518339f None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:43.175718 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbe9c0a-396b-4142-ad9f-d2e65518339f None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:43.246716 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 591/1169] 10.222.0.58 () {60 vars in 1138 bytes} [Thu Aug 25 03:02:43 2022] POST /identity/v3/domains => generated 267 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:02:45.163596 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176b7653-d9c3-43ff-8507-2b9fdf04b67b None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:45.164144 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176b7653-d9c3-43ff-8507-2b9fdf04b67b None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:45.164144 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176b7653-d9c3-43ff-8507-2b9fdf04b67b None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:45.165465 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 579/1170] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:02:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:02:45.170732 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc1283-dbba-4bfb-b297-a1143dee587b None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:45.171220 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc1283-dbba-4bfb-b297-a1143dee587b None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:45.171334 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc1283-dbba-4bfb-b297-a1143dee587b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:45.188780 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-9ebc1283-dbba-4bfb-b297-a1143dee587b None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:02:45.189961 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-9ebc1283-dbba-4bfb-b297-a1143dee587b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:02:45.255056 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 592/1171] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:02:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 03:02:45.528065 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176b7653-d9c3-43ff-8507-2b9fdf04b67b None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:45.548601 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:45.633430 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176b7653-d9c3-43ff-8507-2b9fdf04b67b None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:45.668500 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946e3f5e-a59d-44ac-9e6e-1863d6d234fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:45.669116 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946e3f5e-a59d-44ac-9e6e-1863d6d234fd None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:45.669394 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946e3f5e-a59d-44ac-9e6e-1863d6d234fd None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:45.669666 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946e3f5e-a59d-44ac-9e6e-1863d6d234fd None admin] PATH_INFO: `/v3/domains/d2eb592a4efd443f8de0e83e46b363cb` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:45.679559 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946e3f5e-a59d-44ac-9e6e-1863d6d234fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=d2eb592a4efd443f8de0e83e46b363cb)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:45.680995 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946e3f5e-a59d-44ac-9e6e-1863d6d234fd None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:45.682605 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 580/1172] 10.222.0.58 () {56 vars in 1183 bytes} [Thu Aug 25 03:02:45 2022] GET /identity/v3/domains/d2eb592a4efd443f8de0e83e46b363cb => generated 267 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:02:45.688481 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebc1283-dbba-4bfb-b297-a1143dee587b None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:45.733085 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:45.783822 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebc1283-dbba-4bfb-b297-a1143dee587b None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:45.816013 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51be1a32-5ab1-4229-8b18-f6131bf0a1d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:45.816606 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51be1a32-5ab1-4229-8b18-f6131bf0a1d2 None admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:45.816881 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51be1a32-5ab1-4229-8b18-f6131bf0a1d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:45.817150 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51be1a32-5ab1-4229-8b18-f6131bf0a1d2 None admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:45.817777 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51be1a32-5ab1-4229-8b18-f6131bf0a1d2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:45.819210 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51be1a32-5ab1-4229-8b18-f6131bf0a1d2 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:45.898184 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 593/1173] 10.222.0.58 () {60 vars in 1141 bytes} [Thu Aug 25 03:02:45 2022] POST /identity/v3/projects => generated 361 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:02:47.828702 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64320144-7688-4507-a8c7-994939c6f4f1 None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:47.829573 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64320144-7688-4507-a8c7-994939c6f4f1 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:47.829573 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64320144-7688-4507-a8c7-994939c6f4f1 None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:47.830925 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 581/1174] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:02:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:02:47.836238 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75298e5c-eadf-40ff-a185-aa4c00641455 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:47.836625 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75298e5c-eadf-40ff-a185-aa4c00641455 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:47.836797 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75298e5c-eadf-40ff-a185-aa4c00641455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:47.854393 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-75298e5c-eadf-40ff-a185-aa4c00641455 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:02:47.855570 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-75298e5c-eadf-40ff-a185-aa4c00641455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:02:47.924488 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 594/1175] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:02:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 03:02:48.197380 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64320144-7688-4507-a8c7-994939c6f4f1 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:48.280702 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:48.301554 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64320144-7688-4507-a8c7-994939c6f4f1 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:48.335026 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70688ae0-c60e-403c-b278-3d7487847c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:48.335643 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70688ae0-c60e-403c-b278-3d7487847c95 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:48.335923 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70688ae0-c60e-403c-b278-3d7487847c95 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:48.336199 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70688ae0-c60e-403c-b278-3d7487847c95 None admin] PATH_INFO: `/v3/domains/d2eb592a4efd443f8de0e83e46b363cb` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:48.337294 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70688ae0-c60e-403c-b278-3d7487847c95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=d2eb592a4efd443f8de0e83e46b363cb)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:48.338750 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70688ae0-c60e-403c-b278-3d7487847c95 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:48.340293 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 582/1176] 10.222.0.58 () {56 vars in 1183 bytes} [Thu Aug 25 03:02:48 2022] GET /identity/v3/domains/d2eb592a4efd443f8de0e83e46b363cb => generated 267 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:02:48.346113 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75298e5c-eadf-40ff-a185-aa4c00641455 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:48.372559 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:48.442852 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75298e5c-eadf-40ff-a185-aa4c00641455 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:48.475859 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32901403-11a1-4e8f-80ec-f0b0d6bfab92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:48.476398 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32901403-11a1-4e8f-80ec-f0b0d6bfab92 None admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:48.476680 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32901403-11a1-4e8f-80ec-f0b0d6bfab92 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:48.476956 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32901403-11a1-4e8f-80ec-f0b0d6bfab92 None admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:48.477608 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32901403-11a1-4e8f-80ec-f0b0d6bfab92 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:48.479057 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32901403-11a1-4e8f-80ec-f0b0d6bfab92 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:48.548793 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 595/1177] 10.222.0.58 () {60 vars in 1132 bytes} [Thu Aug 25 03:02:48 2022] POST /identity/v3/users => generated 311 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:02:50.466035 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227fd807-d793-4889-8d6b-44acbc0f577e None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:50.466634 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227fd807-d793-4889-8d6b-44acbc0f577e None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:50.466634 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227fd807-d793-4889-8d6b-44acbc0f577e None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:50.467937 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 583/1178] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:02:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:02:50.473780 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a60cd7-2b62-4d7b-892a-8b254e64f3af None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:50.474190 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a60cd7-2b62-4d7b-892a-8b254e64f3af None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:50.474375 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a60cd7-2b62-4d7b-892a-8b254e64f3af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:50.491873 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-c1a60cd7-2b62-4d7b-892a-8b254e64f3af None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:02:50.493051 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-c1a60cd7-2b62-4d7b-892a-8b254e64f3af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:02:50.556754 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 596/1179] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:02:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 03:02:50.828867 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227fd807-d793-4889-8d6b-44acbc0f577e None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:50.926183 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227fd807-d793-4889-8d6b-44acbc0f577e None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:50.959227 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897146bd-82af-48ea-9a4e-3be7e8d50719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:50.959842 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897146bd-82af-48ea-9a4e-3be7e8d50719 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:50.960121 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897146bd-82af-48ea-9a4e-3be7e8d50719 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:50.960394 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897146bd-82af-48ea-9a4e-3be7e8d50719 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:50.970472 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897146bd-82af-48ea-9a4e-3be7e8d50719 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:02:50.978866 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-897146bd-82af-48ea-9a4e-3be7e8d50719 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 03:02:50.978866 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:02:50.978866 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:02:50.978866 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:02:50.978866 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:02:50.978866 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:02:50.978866 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:02:50.978866 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:02:50.978866 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:02:50.978866 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:02:50.978866 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:02:50.978866 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:02:50.978866 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:02:50.978866 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:02:50.978866 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:02:50.978866 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:02:50.986655 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:02:50.986655 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:02:50.986655 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:02:50.986655 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:02:50.986655 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:02:50.986655 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:02:50.986655 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:02:50.986655 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:02:50.986655 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:02:50.986655 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:02:50.986655 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:02:50.986655 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:02:50.986655 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:02:50.986655 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:02:50.986655 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:02:50.986655 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:02:50.987780 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:02:50.987780 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:02:50.987780 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:02:50.987780 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:02:50.987780 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:02:50.987780 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:02:50.987780 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:02:50.987780 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:02:50.987780 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:02:50.987780 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:02:50.987780 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 03:02:50.987780 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:02:50.987780 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 584/1180] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Aug 25 03:02:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:02:50.988734 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a60cd7-2b62-4d7b-892a-8b254e64f3af None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:51.006006 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:51.090558 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a60cd7-2b62-4d7b-892a-8b254e64f3af None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:51.166114 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395688f4-d0bb-4d31-b0ad-d6ba0a92a646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:51.166737 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395688f4-d0bb-4d31-b0ad-d6ba0a92a646 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:51.167020 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395688f4-d0bb-4d31-b0ad-d6ba0a92a646 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:51.167295 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395688f4-d0bb-4d31-b0ad-d6ba0a92a646 None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:51.167956 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395688f4-d0bb-4d31-b0ad-d6ba0a92a646 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:02:51.168260 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395688f4-d0bb-4d31-b0ad-d6ba0a92a646 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:51.169628 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395688f4-d0bb-4d31-b0ad-d6ba0a92a646 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:51.177967 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 597/1181] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Aug 25 03:02:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:02:51.183998 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897146bd-82af-48ea-9a4e-3be7e8d50719 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:51.193807 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:51.281503 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897146bd-82af-48ea-9a4e-3be7e8d50719 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:51.314750 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c6b322-d83d-4d6f-b1cf-ea7449ec8012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:51.315387 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c6b322-d83d-4d6f-b1cf-ea7449ec8012 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:51.315672 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c6b322-d83d-4d6f-b1cf-ea7449ec8012 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:51.315950 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c6b322-d83d-4d6f-b1cf-ea7449ec8012 None admin] PATH_INFO: `/v3/users/0fc7415f047f48f4809ab708c58786df` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:51.335539 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c6b322-d83d-4d6f-b1cf-ea7449ec8012 None admin] RBAC: Authorizing `identity:get_user(user_id=0fc7415f047f48f4809ab708c58786df)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:51.337005 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c6b322-d83d-4d6f-b1cf-ea7449ec8012 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:51.338720 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 585/1182] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Aug 25 03:02:51 2022] GET /identity/v3/users/0fc7415f047f48f4809ab708c58786df => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:02:51.345272 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395688f4-d0bb-4d31-b0ad-d6ba0a92a646 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:51.419809 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:51.442136 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395688f4-d0bb-4d31-b0ad-d6ba0a92a646 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:51.474543 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d077dd3b-9846-4213-bcb8-e53443b20369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:51.475122 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d077dd3b-9846-4213-bcb8-e53443b20369 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:51.475395 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d077dd3b-9846-4213-bcb8-e53443b20369 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:51.475666 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d077dd3b-9846-4213-bcb8-e53443b20369 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:51.539378 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d077dd3b-9846-4213-bcb8-e53443b20369 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:51.540826 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d077dd3b-9846-4213-bcb8-e53443b20369 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:51.613530 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:51.636293 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 598/1183] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Aug 25 03:02:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 25 03:02:51.642160 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c6b322-d83d-4d6f-b1cf-ea7449ec8012 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:51.701732 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:51.737946 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c6b322-d83d-4d6f-b1cf-ea7449ec8012 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:51.770029 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe1a4ac-dad1-4e9d-960d-aa5447a4fb61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:51.770684 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe1a4ac-dad1-4e9d-960d-aa5447a4fb61 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:51.770969 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe1a4ac-dad1-4e9d-960d-aa5447a4fb61 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:51.771246 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe1a4ac-dad1-4e9d-960d-aa5447a4fb61 None admin] PATH_INFO: `/v3/projects/eaafe89e743648ac981366508ab031c8` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:51.772353 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe1a4ac-dad1-4e9d-960d-aa5447a4fb61 None admin] RBAC: Authorizing `identity:get_project(project_id=eaafe89e743648ac981366508ab031c8)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:51.773818 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe1a4ac-dad1-4e9d-960d-aa5447a4fb61 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:51.775467 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 586/1184] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Aug 25 03:02:51 2022] GET /identity/v3/projects/eaafe89e743648ac981366508ab031c8 => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:02:51.781516 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d077dd3b-9846-4213-bcb8-e53443b20369 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:51.877267 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d077dd3b-9846-4213-bcb8-e53443b20369 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:51.909294 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d134aef-bb22-4b79-a10f-451669084fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:51.909893 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d134aef-bb22-4b79-a10f-451669084fea None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:51.910172 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d134aef-bb22-4b79-a10f-451669084fea None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:51.910478 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d134aef-bb22-4b79-a10f-451669084fea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:51.911196 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d134aef-bb22-4b79-a10f-451669084fea None admin] RBAC: Adding query filter params (role.id=62f19ea272b447ccaab1576cd9bf4fd9, scope.project.id=eaafe89e743648ac981366508ab031c8, user.id=0fc7415f047f48f4809ab708c58786df) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:02:51.911504 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d134aef-bb22-4b79-a10f-451669084fea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:51.912913 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d134aef-bb22-4b79-a10f-451669084fea None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:51.918058 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 599/1185] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Aug 25 03:02:51 2022] GET /identity/v3/role_assignments?user.id=0fc7415f047f48f4809ab708c58786df&scope.project.id=eaafe89e743648ac981366508ab031c8&role.id=62f19ea272b447ccaab1576cd9bf4fd9 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:02:53.793080 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5bbdff-0602-4322-b981-1303b3f33986 None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:53.793606 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5bbdff-0602-4322-b981-1303b3f33986 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:53.793606 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5bbdff-0602-4322-b981-1303b3f33986 None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:53.794976 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 587/1186] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:02:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:02:53.800620 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7349b619-3f57-4042-be9c-639c0c33573e None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:53.801053 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7349b619-3f57-4042-be9c-639c0c33573e None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:53.801155 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7349b619-3f57-4042-be9c-639c0c33573e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:53.819207 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-7349b619-3f57-4042-be9c-639c0c33573e None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:02:53.820374 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-7349b619-3f57-4042-be9c-639c0c33573e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:02:53.853450 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:53.885235 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 600/1187] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:02:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 03:02:54.156889 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5bbdff-0602-4322-b981-1303b3f33986 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:54.254710 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5bbdff-0602-4322-b981-1303b3f33986 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:54.282862 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:54.287555 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4f16a1-4ddb-486b-8675-a3fe541e6766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:54.288176 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4f16a1-4ddb-486b-8675-a3fe541e6766 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:54.288458 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4f16a1-4ddb-486b-8675-a3fe541e6766 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:54.288732 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4f16a1-4ddb-486b-8675-a3fe541e6766 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:54.299554 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4f16a1-4ddb-486b-8675-a3fe541e6766 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:02:54.304266 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-0c4f16a1-4ddb-486b-8675-a3fe541e6766 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 03:02:54.304266 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:02:54.304266 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:02:54.304266 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:02:54.304266 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:02:54.304266 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:02:54.304266 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:02:54.304266 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:02:54.304266 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:02:54.304266 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:02:54.304266 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:02:54.304266 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:02:54.304266 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:02:54.304266 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:02:54.304266 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:02:54.304266 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:02:54.305340 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:02:54.305340 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:02:54.305340 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:02:54.305340 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:02:54.305340 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:02:54.305340 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:02:54.305340 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:02:54.305340 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:02:54.305340 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:02:54.305340 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:02:54.305340 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:02:54.305340 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:02:54.305340 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:02:54.305340 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:02:54.305340 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:02:54.305340 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:02:54.306207 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:02:54.306207 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:02:54.306207 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:02:54.306207 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:02:54.306207 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:02:54.306207 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:02:54.306207 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:02:54.306207 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:02:54.306207 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:02:54.306207 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:02:54.306207 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 03:02:54.306207 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:02:54.306207 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 588/1188] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Aug 25 03:02:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:02:54.311434 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7349b619-3f57-4042-be9c-639c0c33573e None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:54.406819 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7349b619-3f57-4042-be9c-639c0c33573e None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:54.439230 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d587bd0f-5e2e-486c-9212-0a51ce9b814a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:54.439772 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d587bd0f-5e2e-486c-9212-0a51ce9b814a None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:54.440054 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d587bd0f-5e2e-486c-9212-0a51ce9b814a None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:54.440330 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d587bd0f-5e2e-486c-9212-0a51ce9b814a None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:54.440988 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d587bd0f-5e2e-486c-9212-0a51ce9b814a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:02:54.441292 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d587bd0f-5e2e-486c-9212-0a51ce9b814a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:54.442827 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d587bd0f-5e2e-486c-9212-0a51ce9b814a None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:54.450551 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 601/1189] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Aug 25 03:02:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:02:54.457133 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4f16a1-4ddb-486b-8675-a3fe541e6766 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:54.553086 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4f16a1-4ddb-486b-8675-a3fe541e6766 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:54.586715 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca62382f-d2fd-43e6-9399-0183615be4c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:54.587337 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca62382f-d2fd-43e6-9399-0183615be4c3 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:54.587618 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca62382f-d2fd-43e6-9399-0183615be4c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:54.587895 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca62382f-d2fd-43e6-9399-0183615be4c3 None admin] PATH_INFO: `/v3/users/0fc7415f047f48f4809ab708c58786df` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:54.589008 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca62382f-d2fd-43e6-9399-0183615be4c3 None admin] RBAC: Authorizing `identity:get_user(user_id=0fc7415f047f48f4809ab708c58786df)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:54.590469 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca62382f-d2fd-43e6-9399-0183615be4c3 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:54.592026 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 589/1190] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Aug 25 03:02:54 2022] GET /identity/v3/users/0fc7415f047f48f4809ab708c58786df => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:02:54.597720 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d587bd0f-5e2e-486c-9212-0a51ce9b814a None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:54.628957 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:54.694895 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d587bd0f-5e2e-486c-9212-0a51ce9b814a None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:54.726980 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bf9d20-ba8b-4f86-aab4-e1f8ad4f7d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:54.727603 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bf9d20-ba8b-4f86-aab4-e1f8ad4f7d87 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:54.727883 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bf9d20-ba8b-4f86-aab4-e1f8ad4f7d87 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:54.728157 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bf9d20-ba8b-4f86-aab4-e1f8ad4f7d87 None admin] PATH_INFO: `/v3/projects/eaafe89e743648ac981366508ab031c8` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:54.729241 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bf9d20-ba8b-4f86-aab4-e1f8ad4f7d87 None admin] RBAC: Authorizing `identity:get_project(project_id=eaafe89e743648ac981366508ab031c8)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:54.730694 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bf9d20-ba8b-4f86-aab4-e1f8ad4f7d87 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:54.732299 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 602/1191] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Aug 25 03:02:54 2022] GET /identity/v3/projects/eaafe89e743648ac981366508ab031c8 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:02:54.738836 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca62382f-d2fd-43e6-9399-0183615be4c3 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:54.795422 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:54.836109 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca62382f-d2fd-43e6-9399-0183615be4c3 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:54.868142 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf3b2d1-c7d4-4f65-aaa5-df8e3f202b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:54.868823 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf3b2d1-c7d4-4f65-aaa5-df8e3f202b72 None admin] REQUEST_METHOD: `PUT` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:54.869108 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf3b2d1-c7d4-4f65-aaa5-df8e3f202b72 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:54.869394 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf3b2d1-c7d4-4f65-aaa5-df8e3f202b72 None admin] PATH_INFO: `/v3/projects/eaafe89e743648ac981366508ab031c8/users/0fc7415f047f48f4809ab708c58786df/roles/62f19ea272b447ccaab1576cd9bf4fd9` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:54.871299 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf3b2d1-c7d4-4f65-aaa5-df8e3f202b72 None admin] RBAC: Authorizing `identity:create_grant(project_id=eaafe89e743648ac981366508ab031c8, user_id=0fc7415f047f48f4809ab708c58786df, role_id=62f19ea272b447ccaab1576cd9bf4fd9)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:54.872739 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf3b2d1-c7d4-4f65-aaa5-df8e3f202b72 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:54.942070 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 590/1192] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Aug 25 03:02:54 2022] PUT /identity/v3/projects/eaafe89e743648ac981366508ab031c8/users/0fc7415f047f48f4809ab708c58786df/roles/62f19ea272b447ccaab1576cd9bf4fd9 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:02:56.827457 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cca73a-2348-4d6e-9f6f-a99aa210c586 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:56.828012 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cca73a-2348-4d6e-9f6f-a99aa210c586 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:56.828012 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cca73a-2348-4d6e-9f6f-a99aa210c586 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:56.829716 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 603/1193] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:02:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:02:56.835093 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49110817-4180-4d16-a089-340746bebfe4 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:56.835526 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49110817-4180-4d16-a089-340746bebfe4 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:56.835637 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49110817-4180-4d16-a089-340746bebfe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:56.853633 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-49110817-4180-4d16-a089-340746bebfe4 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:02:56.854832 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-49110817-4180-4d16-a089-340746bebfe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:02:56.919988 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 591/1194] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:02:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 03:02:57.191399 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cca73a-2348-4d6e-9f6f-a99aa210c586 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:57.286786 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:57.290496 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cca73a-2348-4d6e-9f6f-a99aa210c586 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:57.323588 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a2401c-5a1c-42cf-b359-aa8ddc7b6580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:57.324208 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a2401c-5a1c-42cf-b359-aa8ddc7b6580 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:57.324489 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a2401c-5a1c-42cf-b359-aa8ddc7b6580 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:57.324762 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a2401c-5a1c-42cf-b359-aa8ddc7b6580 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:57.335536 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a2401c-5a1c-42cf-b359-aa8ddc7b6580 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:02:57.340327 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-b5a2401c-5a1c-42cf-b359-aa8ddc7b6580 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 03:02:57.340327 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:02:57.340327 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:02:57.340327 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:02:57.340327 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:02:57.340327 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:02:57.340327 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:02:57.340327 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:02:57.340327 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:02:57.340327 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:02:57.340327 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:02:57.340327 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:02:57.340327 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:02:57.340327 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:02:57.340327 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:02:57.340327 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:02:57.341421 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:02:57.341421 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:02:57.341421 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:02:57.341421 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:02:57.341421 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:02:57.341421 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:02:57.341421 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:02:57.341421 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:02:57.341421 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:02:57.341421 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:02:57.341421 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:02:57.341421 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:02:57.341421 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:02:57.341421 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:02:57.341421 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:02:57.341421 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:02:57.342631 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:02:57.342631 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:02:57.342631 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:02:57.342631 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:02:57.342631 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:02:57.342631 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:02:57.342631 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:02:57.342631 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:02:57.342631 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:02:57.342631 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:02:57.342631 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 03:02:57.342631 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:02:57.342631 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 604/1195] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Aug 25 03:02:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:02:57.347282 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49110817-4180-4d16-a089-340746bebfe4 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:57.394460 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:57.441937 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49110817-4180-4d16-a089-340746bebfe4 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:57.474635 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae1e436-74bb-4235-bc28-3446e53a107d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:57.475233 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae1e436-74bb-4235-bc28-3446e53a107d None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:57.475514 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae1e436-74bb-4235-bc28-3446e53a107d None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:57.475788 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae1e436-74bb-4235-bc28-3446e53a107d None admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:57.476443 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae1e436-74bb-4235-bc28-3446e53a107d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:02:57.476746 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae1e436-74bb-4235-bc28-3446e53a107d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:57.478108 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae1e436-74bb-4235-bc28-3446e53a107d None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:57.486396 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 592/1196] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Aug 25 03:02:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:02:57.492079 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a2401c-5a1c-42cf-b359-aa8ddc7b6580 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:57.589100 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a2401c-5a1c-42cf-b359-aa8ddc7b6580 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:57.622243 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5f4464-a77f-479a-b688-ccd103a4e5cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:57.622893 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5f4464-a77f-479a-b688-ccd103a4e5cc None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:57.623173 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5f4464-a77f-479a-b688-ccd103a4e5cc None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:57.623445 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5f4464-a77f-479a-b688-ccd103a4e5cc None admin] PATH_INFO: `/v3/users/0fc7415f047f48f4809ab708c58786df` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:57.624543 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5f4464-a77f-479a-b688-ccd103a4e5cc None admin] RBAC: Authorizing `identity:get_user(user_id=0fc7415f047f48f4809ab708c58786df)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:57.625984 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5f4464-a77f-479a-b688-ccd103a4e5cc None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:57.627570 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 605/1197] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Aug 25 03:02:57 2022] GET /identity/v3/users/0fc7415f047f48f4809ab708c58786df => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:02:57.633497 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae1e436-74bb-4235-bc28-3446e53a107d None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:57.730371 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae1e436-74bb-4235-bc28-3446e53a107d None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:57.742527 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:57.762297 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beed7032-c88a-4668-8d47-7597a9b48009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:57.762909 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beed7032-c88a-4668-8d47-7597a9b48009 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:57.763190 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beed7032-c88a-4668-8d47-7597a9b48009 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:57.763466 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beed7032-c88a-4668-8d47-7597a9b48009 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:57.828310 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beed7032-c88a-4668-8d47-7597a9b48009 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:57.830007 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beed7032-c88a-4668-8d47-7597a9b48009 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:57.923529 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 593/1198] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Aug 25 03:02:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 25 03:02:57.929570 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5f4464-a77f-479a-b688-ccd103a4e5cc None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:57.957645 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:58.028109 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5f4464-a77f-479a-b688-ccd103a4e5cc None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:58.062862 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d664737-a651-4bc3-b49a-096fee90f38a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:58.063482 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d664737-a651-4bc3-b49a-096fee90f38a None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:58.063762 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d664737-a651-4bc3-b49a-096fee90f38a None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:58.064035 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d664737-a651-4bc3-b49a-096fee90f38a None admin] PATH_INFO: `/v3/projects/eaafe89e743648ac981366508ab031c8` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:58.065126 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d664737-a651-4bc3-b49a-096fee90f38a None admin] RBAC: Authorizing `identity:get_project(project_id=eaafe89e743648ac981366508ab031c8)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:58.066569 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d664737-a651-4bc3-b49a-096fee90f38a None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:58.068181 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 606/1199] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Aug 25 03:02:57 2022] GET /identity/v3/projects/eaafe89e743648ac981366508ab031c8 => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:02:58.074340 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beed7032-c88a-4668-8d47-7597a9b48009 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:58.088750 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:58.173632 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beed7032-c88a-4668-8d47-7597a9b48009 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:58.206491 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f929d62a-d876-4e49-a84f-19e69da5d44e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:58.207082 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f929d62a-d876-4e49-a84f-19e69da5d44e None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:58.207357 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f929d62a-d876-4e49-a84f-19e69da5d44e None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:58.207625 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f929d62a-d876-4e49-a84f-19e69da5d44e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:58.208343 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f929d62a-d876-4e49-a84f-19e69da5d44e None admin] RBAC: Adding query filter params (role.id=62f19ea272b447ccaab1576cd9bf4fd9, scope.project.id=eaafe89e743648ac981366508ab031c8, user.id=0fc7415f047f48f4809ab708c58786df) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:02:58.208645 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f929d62a-d876-4e49-a84f-19e69da5d44e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:58.210056 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f929d62a-d876-4e49-a84f-19e69da5d44e None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:58.215574 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 594/1200] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Aug 25 03:02:58 2022] GET /identity/v3/role_assignments?user.id=0fc7415f047f48f4809ab708c58786df&scope.project.id=eaafe89e743648ac981366508ab031c8&role.id=62f19ea272b447ccaab1576cd9bf4fd9 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:03:08.058270 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=52 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:03:08.213054 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=62 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:04:10.186770 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e527df1-5aad-4da8-ac70-49f4e596787c None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:10.187369 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e527df1-5aad-4da8-ac70-49f4e596787c None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:10.187369 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e527df1-5aad-4da8-ac70-49f4e596787c None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:10.188662 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 607/1201] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:04:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:04:10.193800 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e395eb8f-9499-42ac-9fa5-21243374204c None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:10.194239 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e395eb8f-9499-42ac-9fa5-21243374204c None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:10.194404 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e395eb8f-9499-42ac-9fa5-21243374204c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:10.213307 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-e395eb8f-9499-42ac-9fa5-21243374204c None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:04:10.214541 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-e395eb8f-9499-42ac-9fa5-21243374204c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:04:10.279909 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 595/1202] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:04:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 03:04:10.551490 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e527df1-5aad-4da8-ac70-49f4e596787c None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:10.653388 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e527df1-5aad-4da8-ac70-49f4e596787c None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:10.686850 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a42ea2-0f12-4896-bff7-80a6134486c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:10.687480 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a42ea2-0f12-4896-bff7-80a6134486c2 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:10.687760 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a42ea2-0f12-4896-bff7-80a6134486c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:10.688032 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a42ea2-0f12-4896-bff7-80a6134486c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:10.697633 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a42ea2-0f12-4896-bff7-80a6134486c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:04:10.699122 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a42ea2-0f12-4896-bff7-80a6134486c2 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:04:10.713978 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-47a42ea2-0f12-4896-bff7-80a6134486c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:04:10.713978 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:04:10.713978 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:04:10.713978 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:04:10.713978 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:04:10.713978 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:04:10.713978 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:04:10.713978 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:04:10.713978 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:04:10.713978 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:04:10.713978 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:04:10.713978 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:04:10.713978 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 03:04:10.713978 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 03:04:10.713978 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 03:04:10.713978 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 03:04:10.715178 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:04:10.715178 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:04:10.715178 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:04:10.715178 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:04:10.715178 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:04:10.715178 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:04:10.715178 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:04:10.715178 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:04:10.715178 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:04:10.715178 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:04:10.715178 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:04:10.715178 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:04:10.715178 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:04:10.715178 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:04:10.715178 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:04:10.715178 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:04:10.715178 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 03:04:10.716168 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 03:04:10.716168 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:04:10.716168 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:04:10.716168 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 608/1203] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:04:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:04:10.721129 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e395eb8f-9499-42ac-9fa5-21243374204c None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:10.817327 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e395eb8f-9499-42ac-9fa5-21243374204c None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:10.844223 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:04:10.849971 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c2ac3f-53d6-4244-8f66-242467ce287b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:10.851486 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c2ac3f-53d6-4244-8f66-242467ce287b None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:10.851486 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c2ac3f-53d6-4244-8f66-242467ce287b None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:10.851486 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c2ac3f-53d6-4244-8f66-242467ce287b None admin] PATH_INFO: `/v3/domains` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:10.851805 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c2ac3f-53d6-4244-8f66-242467ce287b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:04:10.852133 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c2ac3f-53d6-4244-8f66-242467ce287b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:04:10.853494 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c2ac3f-53d6-4244-8f66-242467ce287b None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:04:10.864644 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 596/1204] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Aug 25 03:04:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:04:10.871122 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a42ea2-0f12-4896-bff7-80a6134486c2 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:10.923914 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:04:10.969800 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a42ea2-0f12-4896-bff7-80a6134486c2 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:11.002885 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e645084-0d60-43df-ac5e-8261d9200511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:11.010406 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e645084-0d60-43df-ac5e-8261d9200511 None admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:11.010406 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e645084-0d60-43df-ac5e-8261d9200511 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:11.010406 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e645084-0d60-43df-ac5e-8261d9200511 None admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:11.010406 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e645084-0d60-43df-ac5e-8261d9200511 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:04:11.010406 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e645084-0d60-43df-ac5e-8261d9200511 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:04:11.126760 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 609/1205] 10.222.0.58 () {60 vars in 1132 bytes} [Thu Aug 25 03:04:10 2022] POST /identity/v3/users => generated 251 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:04:13.084178 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28477d78-2ef4-466c-bb4b-314afd2facd8 None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:13.084722 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28477d78-2ef4-466c-bb4b-314afd2facd8 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:13.084722 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28477d78-2ef4-466c-bb4b-314afd2facd8 None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:13.086061 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 597/1206] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:04:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:04:13.091624 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96062607-4392-4228-932e-aa44ca6e40ed None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:13.096430 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96062607-4392-4228-932e-aa44ca6e40ed None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:13.096430 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96062607-4392-4228-932e-aa44ca6e40ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:13.110438 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-96062607-4392-4228-932e-aa44ca6e40ed None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:04:13.111630 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-96062607-4392-4228-932e-aa44ca6e40ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:04:13.177132 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 610/1207] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:04:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 03:04:13.446078 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28477d78-2ef4-466c-bb4b-314afd2facd8 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:13.547329 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28477d78-2ef4-466c-bb4b-314afd2facd8 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:13.579962 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902cb494-5b25-41e3-b997-a07195e63f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:13.580864 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902cb494-5b25-41e3-b997-a07195e63f2f None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:13.581146 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902cb494-5b25-41e3-b997-a07195e63f2f None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:13.581423 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902cb494-5b25-41e3-b997-a07195e63f2f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:13.592317 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902cb494-5b25-41e3-b997-a07195e63f2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:04:13.597043 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-902cb494-5b25-41e3-b997-a07195e63f2f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 25 03:04:13.597043 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:04:13.597043 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:04:13.597043 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:04:13.597043 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:04:13.597043 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:04:13.597043 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:04:13.597043 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:04:13.597043 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:04:13.597043 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:04:13.597043 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:04:13.597043 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:04:13.597043 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:04:13.597043 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:04:13.597043 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:04:13.597043 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:04:13.598208 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:04:13.598208 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:04:13.598208 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:04:13.598208 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:04:13.598208 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:04:13.598208 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:04:13.598208 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:04:13.598208 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:04:13.598208 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:04:13.598208 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:04:13.598208 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:04:13.598208 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:04:13.598208 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:04:13.598208 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:04:13.598208 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:04:13.598208 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:04:13.599452 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:04:13.599452 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:04:13.599452 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:04:13.599452 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:04:13.599452 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:04:13.599452 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:04:13.599452 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:04:13.599452 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:04:13.599452 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:04:13.599452 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:04:13.599452 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 25 03:04:13.599452 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:04:13.599452 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 598/1208] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Aug 25 03:04:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:04:13.604126 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96062607-4392-4228-932e-aa44ca6e40ed None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:13.645463 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:04:13.703809 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96062607-4392-4228-932e-aa44ca6e40ed None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:13.736897 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c20a1e-39ec-48ae-b89b-699a8ca5e000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:13.737537 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c20a1e-39ec-48ae-b89b-699a8ca5e000 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:13.737811 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c20a1e-39ec-48ae-b89b-699a8ca5e000 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:13.738080 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c20a1e-39ec-48ae-b89b-699a8ca5e000 None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:13.738765 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c20a1e-39ec-48ae-b89b-699a8ca5e000 None admin] RBAC: Adding query filter params (name=service) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:04:13.746197 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c20a1e-39ec-48ae-b89b-699a8ca5e000 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:04:13.746197 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c20a1e-39ec-48ae-b89b-699a8ca5e000 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:04:13.748169 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 611/1209] 10.222.0.58 () {56 vars in 1103 bytes} [Thu Aug 25 03:04:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:04:13.754201 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902cb494-5b25-41e3-b997-a07195e63f2f None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:13.807824 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:04:13.856451 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902cb494-5b25-41e3-b997-a07195e63f2f None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:13.888720 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c139016e-33a4-486a-9164-0e7fc8d6f3fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:13.889331 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c139016e-33a4-486a-9164-0e7fc8d6f3fe None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:13.889607 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c139016e-33a4-486a-9164-0e7fc8d6f3fe None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:13.889875 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c139016e-33a4-486a-9164-0e7fc8d6f3fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:13.900469 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c139016e-33a4-486a-9164-0e7fc8d6f3fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:04:13.908174 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c139016e-33a4-486a-9164-0e7fc8d6f3fe None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:04:13.915905 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-c139016e-33a4-486a-9164-0e7fc8d6f3fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:04:13.915905 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:04:13.915905 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:04:13.915905 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:04:13.915905 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:04:13.915905 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:04:13.915905 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:04:13.915905 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:04:13.915905 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:04:13.915905 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:04:13.915905 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:04:13.915905 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:04:13.915905 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 03:04:13.915905 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 03:04:13.915905 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 03:04:13.915905 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 03:04:13.917090 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:04:13.917090 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:04:13.917090 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:04:13.917090 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:04:13.917090 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:04:13.917090 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:04:13.917090 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:04:13.917090 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:04:13.917090 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:04:13.917090 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:04:13.917090 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:04:13.917090 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:04:13.917090 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:04:13.917090 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:04:13.917090 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:04:13.917090 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:04:13.917090 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 03:04:13.918215 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 03:04:13.918215 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:04:13.918215 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:04:13.918215 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 599/1210] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:04:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:04:13.923257 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c20a1e-39ec-48ae-b89b-699a8ca5e000 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:14.023162 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:04:14.026053 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c20a1e-39ec-48ae-b89b-699a8ca5e000 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:14.058607 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ac0867-34da-4c5b-8f58-a0c8b4941a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:14.059207 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ac0867-34da-4c5b-8f58-a0c8b4941a88 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:14.059486 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ac0867-34da-4c5b-8f58-a0c8b4941a88 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:14.059757 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ac0867-34da-4c5b-8f58-a0c8b4941a88 None admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:14.060391 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ac0867-34da-4c5b-8f58-a0c8b4941a88 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:04:14.060692 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ac0867-34da-4c5b-8f58-a0c8b4941a88 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:04:14.062059 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ac0867-34da-4c5b-8f58-a0c8b4941a88 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:04:14.076060 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 612/1211] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Aug 25 03:04:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:04:14.082434 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c139016e-33a4-486a-9164-0e7fc8d6f3fe None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:14.180381 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c139016e-33a4-486a-9164-0e7fc8d6f3fe None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:14.181899 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:04:14.214907 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0833b64-ee94-4ba2-b042-cf2dba61ba3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:14.215533 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0833b64-ee94-4ba2-b042-cf2dba61ba3c None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:14.215812 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0833b64-ee94-4ba2-b042-cf2dba61ba3c None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:14.216084 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0833b64-ee94-4ba2-b042-cf2dba61ba3c None admin] PATH_INFO: `/v3/users/placement` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:14.222804 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0833b64-ee94-4ba2-b042-cf2dba61ba3c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:04:14.224250 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0833b64-ee94-4ba2-b042-cf2dba61ba3c None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:04:14.235705 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-f0833b64-ee94-4ba2-b042-cf2dba61ba3c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 25 03:04:14.235705 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:04:14.235705 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:04:14.235705 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:04:14.235705 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:04:14.235705 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:04:14.235705 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:04:14.235705 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:04:14.235705 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:04:14.235705 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:04:14.235705 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:04:14.235705 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:04:14.235705 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 25 03:04:14.235705 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 25 03:04:14.235705 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 25 03:04:14.235705 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 25 03:04:14.236892 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:04:14.236892 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:04:14.236892 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 25 03:04:14.236892 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 03:04:14.236892 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 25 03:04:14.236892 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 03:04:14.236892 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:04:14.236892 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:04:14.236892 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:04:14.236892 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:04:14.236892 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:04:14.236892 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:04:14.236892 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:04:14.236892 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:04:14.236892 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:04:14.236892 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:04:14.236892 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:04:14.236892 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:04:14.238057 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:04:14.238057 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:04:14.238057 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 25 03:04:14.238057 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 25 03:04:14.238057 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 25 03:04:14.238057 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 25 03:04:14.238057 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 25 03:04:14.238057 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 25 03:04:14.238057 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 25 03:04:14.238057 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:04:14.238057 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 600/1212] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:04:14 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:04:14.242823 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ac0867-34da-4c5b-8f58-a0c8b4941a88 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:14.340101 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ac0867-34da-4c5b-8f58-a0c8b4941a88 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:14.373330 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152efbed-1ea4-4b4a-b8cd-7a5516350574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:14.373925 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152efbed-1ea4-4b4a-b8cd-7a5516350574 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:14.374209 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152efbed-1ea4-4b4a-b8cd-7a5516350574 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:14.374511 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152efbed-1ea4-4b4a-b8cd-7a5516350574 None admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:14.375237 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152efbed-1ea4-4b4a-b8cd-7a5516350574 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:04:14.375543 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152efbed-1ea4-4b4a-b8cd-7a5516350574 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:04:14.376972 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152efbed-1ea4-4b4a-b8cd-7a5516350574 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:04:14.394528 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 613/1213] 10.222.0.58 () {56 vars in 1143 bytes} [Thu Aug 25 03:04:14 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:04:14.400455 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0833b64-ee94-4ba2-b042-cf2dba61ba3c None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:14.498899 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0833b64-ee94-4ba2-b042-cf2dba61ba3c None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:14.531569 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d13ac4-497b-4244-8a5c-321d9474f7c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:14.532174 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d13ac4-497b-4244-8a5c-321d9474f7c3 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:14.532455 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d13ac4-497b-4244-8a5c-321d9474f7c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:14.532729 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d13ac4-497b-4244-8a5c-321d9474f7c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:14.562829 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:04:14.599943 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d13ac4-497b-4244-8a5c-321d9474f7c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:04:14.601368 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d13ac4-497b-4244-8a5c-321d9474f7c3 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:04:14.700733 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 601/1214] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Aug 25 03:04:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 25 03:04:14.706906 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152efbed-1ea4-4b4a-b8cd-7a5516350574 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:14.726519 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:04:14.805045 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152efbed-1ea4-4b4a-b8cd-7a5516350574 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:14.838759 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840822d4-d76a-4fed-89d7-215e68977a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:14.839384 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840822d4-d76a-4fed-89d7-215e68977a38 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:14.839665 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840822d4-d76a-4fed-89d7-215e68977a38 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:14.839935 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840822d4-d76a-4fed-89d7-215e68977a38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:14.849876 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840822d4-d76a-4fed-89d7-215e68977a38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:04:14.851439 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840822d4-d76a-4fed-89d7-215e68977a38 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:04:14.865382 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-840822d4-d76a-4fed-89d7-215e68977a38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:04:14.865382 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:04:14.865382 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:04:14.865382 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:04:14.865382 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:04:14.865382 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:04:14.865382 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:04:14.865382 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:04:14.865382 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:04:14.865382 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:04:14.865382 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:04:14.865382 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:04:14.865382 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 03:04:14.865382 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 03:04:14.865382 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 03:04:14.865382 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 03:04:14.866395 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:04:14.866395 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:04:14.866395 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:04:14.866395 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:04:14.866395 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:04:14.866395 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:04:14.866395 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:04:14.866395 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:04:14.866395 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:04:14.866395 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:04:14.866395 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:04:14.866395 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:04:14.866395 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:04:14.866395 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:04:14.866395 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:04:14.866395 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:04:14.866395 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 03:04:14.867384 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 03:04:14.867384 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:04:14.867384 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:04:14.867384 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 614/1215] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:04:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:04:14.872914 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d13ac4-497b-4244-8a5c-321d9474f7c3 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:14.931879 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:04:14.971327 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d13ac4-497b-4244-8a5c-321d9474f7c3 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:15.004038 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebace94-33de-4d56-ae26-7d4b8aeb95fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:15.004647 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebace94-33de-4d56-ae26-7d4b8aeb95fb None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:15.004926 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebace94-33de-4d56-ae26-7d4b8aeb95fb None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:15.005203 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebace94-33de-4d56-ae26-7d4b8aeb95fb None admin] PATH_INFO: `/v3/domains` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:15.005896 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebace94-33de-4d56-ae26-7d4b8aeb95fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:04:15.006183 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebace94-33de-4d56-ae26-7d4b8aeb95fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:04:15.007582 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebace94-33de-4d56-ae26-7d4b8aeb95fb None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:04:15.019573 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 602/1216] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Aug 25 03:04:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:04:15.025414 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840822d4-d76a-4fed-89d7-215e68977a38 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:15.093530 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:04:15.124921 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840822d4-d76a-4fed-89d7-215e68977a38 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:15.157141 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dc78f8-eb62-4409-ba5d-e556c8181ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:15.157765 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dc78f8-eb62-4409-ba5d-e556c8181ef5 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:15.158046 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dc78f8-eb62-4409-ba5d-e556c8181ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:15.158346 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dc78f8-eb62-4409-ba5d-e556c8181ef5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:15.165638 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dc78f8-eb62-4409-ba5d-e556c8181ef5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:04:15.167648 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dc78f8-eb62-4409-ba5d-e556c8181ef5 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:04:15.179073 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-e7dc78f8-eb62-4409-ba5d-e556c8181ef5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 03:04:15.179073 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:04:15.179073 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:04:15.179073 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:04:15.179073 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:04:15.179073 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:04:15.179073 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:04:15.179073 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:04:15.179073 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:04:15.179073 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:04:15.179073 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:04:15.179073 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:04:15.179073 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 25 03:04:15.179073 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 25 03:04:15.179073 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 25 03:04:15.180157 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 25 03:04:15.180157 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:04:15.180157 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:04:15.180157 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:04:15.180157 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:04:15.180157 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:04:15.180157 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:04:15.180157 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:04:15.180157 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:04:15.180157 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:04:15.180157 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:04:15.180157 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:04:15.180157 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:04:15.180157 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:04:15.180157 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:04:15.180157 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:04:15.180157 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:04:15.181437 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 25 03:04:15.181437 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 25 03:04:15.181437 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 25 03:04:15.181437 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 25 03:04:15.181437 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 25 03:04:15.181437 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 25 03:04:15.181437 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 03:04:15.181437 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:04:15.181437 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 615/1217] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Aug 25 03:04:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:04:15.186606 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebace94-33de-4d56-ae26-7d4b8aeb95fb None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:15.282638 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebace94-33de-4d56-ae26-7d4b8aeb95fb None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:15.304611 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:04:15.316132 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c2e88f-cd66-491b-a845-201cfbded085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:15.316735 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c2e88f-cd66-491b-a845-201cfbded085 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:15.317020 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c2e88f-cd66-491b-a845-201cfbded085 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:15.317294 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c2e88f-cd66-491b-a845-201cfbded085 None admin] PATH_INFO: `/v3/projects` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:15.317982 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c2e88f-cd66-491b-a845-201cfbded085 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:04:15.318298 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c2e88f-cd66-491b-a845-201cfbded085 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:04:15.319736 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c2e88f-cd66-491b-a845-201cfbded085 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:04:15.330989 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 603/1218] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Aug 25 03:04:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:04:15.337084 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dc78f8-eb62-4409-ba5d-e556c8181ef5 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:15.440605 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dc78f8-eb62-4409-ba5d-e556c8181ef5 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:15.468178 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:04:15.473757 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957cd9f2-2872-486a-88b8-c7a85c3a3ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:15.474421 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957cd9f2-2872-486a-88b8-c7a85c3a3ebf None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:15.474693 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957cd9f2-2872-486a-88b8-c7a85c3a3ebf None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:15.474960 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957cd9f2-2872-486a-88b8-c7a85c3a3ebf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:15.475667 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957cd9f2-2872-486a-88b8-c7a85c3a3ebf None admin] RBAC: Adding query filter params (role.id=a93ddb1fd6f6459fb46a17573b821422, scope.project.id=9fd4be43faec4e4191e0e4ec4a4588c3, user.id=2b692edd3e8c41289cc42e1afb4a6dd8) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:04:15.475969 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957cd9f2-2872-486a-88b8-c7a85c3a3ebf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:04:15.477379 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957cd9f2-2872-486a-88b8-c7a85c3a3ebf None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:04:15.483183 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 616/1219] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Aug 25 03:04:15 2022] GET /identity/v3/role_assignments?user.id=2b692edd3e8c41289cc42e1afb4a6dd8&scope.project.id=9fd4be43faec4e4191e0e4ec4a4588c3&role.id=a93ddb1fd6f6459fb46a17573b821422 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:04:17.353115 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0bdbb1-46eb-4d70-bbd5-d63745d5dc4b None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:17.353681 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0bdbb1-46eb-4d70-bbd5-d63745d5dc4b None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:17.353681 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0bdbb1-46eb-4d70-bbd5-d63745d5dc4b None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:17.355023 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 604/1220] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:04:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:04:17.360019 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fadb15-471c-4db9-8205-c4aed32ca706 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:17.360525 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fadb15-471c-4db9-8205-c4aed32ca706 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:17.360652 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fadb15-471c-4db9-8205-c4aed32ca706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:17.378197 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-f5fadb15-471c-4db9-8205-c4aed32ca706 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:04:17.379427 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-f5fadb15-471c-4db9-8205-c4aed32ca706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:04:17.444127 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 617/1221] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:04:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 03:04:17.712665 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0bdbb1-46eb-4d70-bbd5-d63745d5dc4b None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:17.814972 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0bdbb1-46eb-4d70-bbd5-d63745d5dc4b None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:17.848971 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce51ab2c-fffe-4b6d-b71c-58156a252409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:17.849595 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce51ab2c-fffe-4b6d-b71c-58156a252409 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:17.849875 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce51ab2c-fffe-4b6d-b71c-58156a252409 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:17.850181 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce51ab2c-fffe-4b6d-b71c-58156a252409 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:17.860123 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce51ab2c-fffe-4b6d-b71c-58156a252409 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:04:17.864850 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-ce51ab2c-fffe-4b6d-b71c-58156a252409 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 25 03:04:17.864850 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:04:17.864850 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:04:17.864850 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:04:17.864850 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:04:17.864850 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:04:17.864850 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:04:17.864850 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:04:17.864850 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:04:17.864850 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:04:17.864850 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:04:17.864850 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:04:17.864850 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:04:17.864850 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:04:17.864850 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:04:17.864850 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:04:17.865984 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:04:17.865984 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:04:17.865984 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:04:17.865984 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:04:17.865984 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:04:17.865984 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:04:17.865984 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:04:17.865984 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:04:17.865984 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:04:17.865984 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:04:17.865984 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:04:17.865984 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:04:17.865984 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:04:17.865984 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:04:17.865984 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:04:17.865984 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:04:17.867227 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:04:17.867227 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:04:17.867227 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:04:17.867227 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:04:17.867227 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:04:17.867227 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:04:17.867227 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:04:17.867227 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:04:17.867227 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:04:17.867227 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:04:17.867227 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 25 03:04:17.867227 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:04:17.867227 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 605/1222] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Aug 25 03:04:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:04:17.872425 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fadb15-471c-4db9-8205-c4aed32ca706 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:17.969662 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fadb15-471c-4db9-8205-c4aed32ca706 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:17.992955 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:04:18.003572 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08ac638-8653-43a8-9e11-9eb1214f0dea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:18.004176 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08ac638-8653-43a8-9e11-9eb1214f0dea None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:18.004454 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08ac638-8653-43a8-9e11-9eb1214f0dea None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:18.004726 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08ac638-8653-43a8-9e11-9eb1214f0dea None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:18.005382 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08ac638-8653-43a8-9e11-9eb1214f0dea None admin] RBAC: Adding query filter params (name=service) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:04:18.005683 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08ac638-8653-43a8-9e11-9eb1214f0dea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:04:18.007146 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08ac638-8653-43a8-9e11-9eb1214f0dea None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:04:18.014782 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 618/1223] 10.222.0.58 () {56 vars in 1103 bytes} [Thu Aug 25 03:04:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:04:18.020618 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce51ab2c-fffe-4b6d-b71c-58156a252409 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:18.064621 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:04:18.118423 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce51ab2c-fffe-4b6d-b71c-58156a252409 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:18.150844 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2869e5f-d70a-483b-b172-779f2a68408a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:18.151456 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2869e5f-d70a-483b-b172-779f2a68408a None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:18.151739 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2869e5f-d70a-483b-b172-779f2a68408a None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:18.152009 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2869e5f-d70a-483b-b172-779f2a68408a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:18.161500 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2869e5f-d70a-483b-b172-779f2a68408a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:04:18.162963 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2869e5f-d70a-483b-b172-779f2a68408a None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:04:18.177400 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-d2869e5f-d70a-483b-b172-779f2a68408a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:04:18.177400 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:04:18.177400 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:04:18.177400 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:04:18.177400 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:04:18.177400 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:04:18.177400 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:04:18.177400 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:04:18.177400 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:04:18.177400 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:04:18.177400 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:04:18.177400 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:04:18.177400 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 03:04:18.177400 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 03:04:18.177400 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 03:04:18.177400 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 03:04:18.185561 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:04:18.185561 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:04:18.185561 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:04:18.185561 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:04:18.185561 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:04:18.185561 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:04:18.185561 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:04:18.185561 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:04:18.185561 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:04:18.185561 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:04:18.185561 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:04:18.185561 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:04:18.185561 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:04:18.185561 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:04:18.185561 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:04:18.185561 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:04:18.185561 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 03:04:18.186556 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 03:04:18.186556 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:04:18.186556 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:04:18.186556 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 606/1224] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:04:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:04:18.186791 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08ac638-8653-43a8-9e11-9eb1214f0dea None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:18.282082 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08ac638-8653-43a8-9e11-9eb1214f0dea None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:18.315825 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaf3eb4-f2c7-4fbb-a352-470ea455ef4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:18.316431 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaf3eb4-f2c7-4fbb-a352-470ea455ef4e None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:18.316714 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaf3eb4-f2c7-4fbb-a352-470ea455ef4e None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:18.316986 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaf3eb4-f2c7-4fbb-a352-470ea455ef4e None admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:18.317617 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaf3eb4-f2c7-4fbb-a352-470ea455ef4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:04:18.317919 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaf3eb4-f2c7-4fbb-a352-470ea455ef4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:04:18.319313 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaf3eb4-f2c7-4fbb-a352-470ea455ef4e None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:04:18.331148 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 619/1225] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Aug 25 03:04:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:04:18.336991 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2869e5f-d70a-483b-b172-779f2a68408a None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:18.430375 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:04:18.436798 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2869e5f-d70a-483b-b172-779f2a68408a None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:18.469740 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80316b17-de82-4533-826f-3c3b456288d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:18.470556 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80316b17-de82-4533-826f-3c3b456288d3 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:18.470773 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80316b17-de82-4533-826f-3c3b456288d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:18.471045 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80316b17-de82-4533-826f-3c3b456288d3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:18.477714 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80316b17-de82-4533-826f-3c3b456288d3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:04:18.479167 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80316b17-de82-4533-826f-3c3b456288d3 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:04:18.490666 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-80316b17-de82-4533-826f-3c3b456288d3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 25 03:04:18.490666 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:04:18.490666 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:04:18.490666 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:04:18.490666 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:04:18.490666 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:04:18.490666 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:04:18.490666 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:04:18.490666 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:04:18.490666 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:04:18.490666 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:04:18.490666 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:04:18.490666 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 25 03:04:18.490666 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 25 03:04:18.490666 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 25 03:04:18.490666 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 25 03:04:18.491838 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:04:18.491838 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:04:18.491838 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 25 03:04:18.491838 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 03:04:18.491838 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 25 03:04:18.491838 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 03:04:18.491838 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:04:18.491838 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:04:18.491838 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:04:18.491838 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:04:18.491838 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:04:18.491838 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:04:18.491838 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:04:18.491838 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:04:18.491838 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:04:18.491838 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:04:18.491838 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:04:18.491838 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:04:18.492964 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:04:18.492964 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:04:18.492964 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 25 03:04:18.492964 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 25 03:04:18.492964 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 25 03:04:18.492964 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 25 03:04:18.492964 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 25 03:04:18.492964 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 25 03:04:18.492964 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 25 03:04:18.492964 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:04:18.492964 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 607/1226] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:04:18 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:04:18.497812 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaf3eb4-f2c7-4fbb-a352-470ea455ef4e None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:18.544070 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:04:18.597892 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaf3eb4-f2c7-4fbb-a352-470ea455ef4e None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:18.630918 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2160bc0-dfcc-470b-9231-69c37f6a7774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:18.631545 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2160bc0-dfcc-470b-9231-69c37f6a7774 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:18.631816 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2160bc0-dfcc-470b-9231-69c37f6a7774 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:18.632087 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2160bc0-dfcc-470b-9231-69c37f6a7774 None admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:18.632801 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2160bc0-dfcc-470b-9231-69c37f6a7774 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:04:18.633099 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2160bc0-dfcc-470b-9231-69c37f6a7774 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:04:18.634551 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2160bc0-dfcc-470b-9231-69c37f6a7774 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:04:18.652619 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 620/1227] 10.222.0.58 () {56 vars in 1143 bytes} [Thu Aug 25 03:04:18 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:04:18.658378 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80316b17-de82-4533-826f-3c3b456288d3 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:18.755227 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80316b17-de82-4533-826f-3c3b456288d3 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:18.788806 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43d0c4b-31f9-43ca-b292-5263794636ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:18.789461 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43d0c4b-31f9-43ca-b292-5263794636ba None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:18.789742 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43d0c4b-31f9-43ca-b292-5263794636ba None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:18.790014 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43d0c4b-31f9-43ca-b292-5263794636ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:18.800726 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43d0c4b-31f9-43ca-b292-5263794636ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:04:18.802153 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43d0c4b-31f9-43ca-b292-5263794636ba None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:04:18.815850 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-f43d0c4b-31f9-43ca-b292-5263794636ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:04:18.815850 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:04:18.815850 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:04:18.815850 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:04:18.815850 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:04:18.815850 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:04:18.815850 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:04:18.815850 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:04:18.815850 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:04:18.815850 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:04:18.815850 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:04:18.815850 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:04:18.815850 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 03:04:18.815850 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 03:04:18.815850 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 03:04:18.815850 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 03:04:18.817091 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:04:18.817091 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:04:18.817091 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:04:18.817091 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:04:18.817091 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:04:18.817091 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:04:18.817091 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:04:18.817091 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:04:18.817091 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:04:18.817091 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:04:18.817091 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:04:18.817091 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:04:18.817091 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:04:18.817091 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:04:18.817091 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:04:18.817091 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:04:18.817091 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 03:04:18.818325 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 03:04:18.818325 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:04:18.818325 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:04:18.818325 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 608/1228] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:04:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:04:18.822916 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2160bc0-dfcc-470b-9231-69c37f6a7774 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:18.924842 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2160bc0-dfcc-470b-9231-69c37f6a7774 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:18.927196 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:04:18.957865 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185ff2ee-4b0c-4315-af7e-0743acee9b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:18.958505 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185ff2ee-4b0c-4315-af7e-0743acee9b64 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:18.958790 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185ff2ee-4b0c-4315-af7e-0743acee9b64 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:18.959063 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185ff2ee-4b0c-4315-af7e-0743acee9b64 None admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:18.959702 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185ff2ee-4b0c-4315-af7e-0743acee9b64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:04:18.960004 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185ff2ee-4b0c-4315-af7e-0743acee9b64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:04:18.961359 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185ff2ee-4b0c-4315-af7e-0743acee9b64 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:04:18.973005 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 621/1229] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Aug 25 03:04:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:04:18.979742 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43d0c4b-31f9-43ca-b292-5263794636ba None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:18.988369 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:04:19.074742 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43d0c4b-31f9-43ca-b292-5263794636ba None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:19.108486 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4002bbca-2651-47c5-83ed-804787ead7a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:19.109223 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4002bbca-2651-47c5-83ed-804787ead7a0 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:19.109508 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4002bbca-2651-47c5-83ed-804787ead7a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:19.109781 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4002bbca-2651-47c5-83ed-804787ead7a0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:19.116243 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4002bbca-2651-47c5-83ed-804787ead7a0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:04:19.118660 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4002bbca-2651-47c5-83ed-804787ead7a0 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:04:19.129227 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-4002bbca-2651-47c5-83ed-804787ead7a0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 03:04:19.129227 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:04:19.129227 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:04:19.129227 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:04:19.129227 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:04:19.129227 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:04:19.129227 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:04:19.129227 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:04:19.129227 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:04:19.129227 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:04:19.129227 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:04:19.129227 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:04:19.129227 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 25 03:04:19.129227 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 25 03:04:19.129227 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 25 03:04:19.135258 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 25 03:04:19.135258 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:04:19.135258 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:04:19.135258 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:04:19.135258 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:04:19.135258 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:04:19.135258 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:04:19.135258 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:04:19.135258 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:04:19.135258 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:04:19.135258 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:04:19.135258 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:04:19.135258 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:04:19.135258 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:04:19.135258 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:04:19.135258 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:04:19.135258 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:04:19.136922 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 25 03:04:19.136922 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 25 03:04:19.136922 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 25 03:04:19.136922 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 25 03:04:19.136922 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 25 03:04:19.136922 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 25 03:04:19.136922 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 03:04:19.136922 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:04:19.136922 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 609/1230] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Aug 25 03:04:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:04:19.137616 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185ff2ee-4b0c-4315-af7e-0743acee9b64 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:19.233797 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185ff2ee-4b0c-4315-af7e-0743acee9b64 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:19.267166 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf1b5f7-e7ca-492c-a5c3-bd79e6889af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:19.267772 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf1b5f7-e7ca-492c-a5c3-bd79e6889af1 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:19.268056 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf1b5f7-e7ca-492c-a5c3-bd79e6889af1 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:19.268333 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf1b5f7-e7ca-492c-a5c3-bd79e6889af1 None admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:19.269022 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf1b5f7-e7ca-492c-a5c3-bd79e6889af1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:04:19.269330 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf1b5f7-e7ca-492c-a5c3-bd79e6889af1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:04:19.270800 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf1b5f7-e7ca-492c-a5c3-bd79e6889af1 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:04:19.282390 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 622/1231] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Aug 25 03:04:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:04:19.288578 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4002bbca-2651-47c5-83ed-804787ead7a0 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:19.353997 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:04:19.387463 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4002bbca-2651-47c5-83ed-804787ead7a0 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:19.419897 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a016fbe-b98c-405e-a987-e6ad1b33f3d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:19.420536 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a016fbe-b98c-405e-a987-e6ad1b33f3d1 None admin] REQUEST_METHOD: `PUT` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:19.420858 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a016fbe-b98c-405e-a987-e6ad1b33f3d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:19.421135 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a016fbe-b98c-405e-a987-e6ad1b33f3d1 None admin] PATH_INFO: `/v3/projects/9fd4be43faec4e4191e0e4ec4a4588c3/users/2b692edd3e8c41289cc42e1afb4a6dd8/roles/a93ddb1fd6f6459fb46a17573b821422` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:19.435524 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a016fbe-b98c-405e-a987-e6ad1b33f3d1 None admin] RBAC: Authorizing `identity:create_grant(project_id=9fd4be43faec4e4191e0e4ec4a4588c3, user_id=2b692edd3e8c41289cc42e1afb4a6dd8, role_id=a93ddb1fd6f6459fb46a17573b821422)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:04:19.436979 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a016fbe-b98c-405e-a987-e6ad1b33f3d1 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:04:19.509310 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 610/1232] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Aug 25 03:04:19 2022] PUT /identity/v3/projects/9fd4be43faec4e4191e0e4ec4a4588c3/users/2b692edd3e8c41289cc42e1afb4a6dd8/roles/a93ddb1fd6f6459fb46a17573b821422 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:04:21.409820 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bd46be-6973-43c4-a303-8f6bfe6adec3 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:21.410408 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bd46be-6973-43c4-a303-8f6bfe6adec3 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:21.410408 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bd46be-6973-43c4-a303-8f6bfe6adec3 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:21.411753 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 623/1233] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:04:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:04:21.417629 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bda948-58ce-4bf7-9bae-48b67076bdb2 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:21.418102 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bda948-58ce-4bf7-9bae-48b67076bdb2 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:21.418197 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bda948-58ce-4bf7-9bae-48b67076bdb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:21.435825 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-73bda948-58ce-4bf7-9bae-48b67076bdb2 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:04:21.437002 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-73bda948-58ce-4bf7-9bae-48b67076bdb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:04:21.502623 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 611/1234] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:04:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 03:04:21.774625 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bd46be-6973-43c4-a303-8f6bfe6adec3 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:21.800335 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:04:21.877117 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bd46be-6973-43c4-a303-8f6bfe6adec3 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:21.911097 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9997cb-57c0-45bc-9d36-fad7f49faffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:21.911722 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9997cb-57c0-45bc-9d36-fad7f49faffa None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:21.912001 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9997cb-57c0-45bc-9d36-fad7f49faffa None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:21.912276 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9997cb-57c0-45bc-9d36-fad7f49faffa None admin] PATH_INFO: `/v3/roles/service` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:21.922377 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9997cb-57c0-45bc-9d36-fad7f49faffa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:04:21.927026 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-0f9997cb-57c0-45bc-9d36-fad7f49faffa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 25 03:04:21.927026 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:04:21.927026 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:04:21.927026 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:04:21.927026 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:04:21.927026 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:04:21.927026 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:04:21.927026 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:04:21.927026 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:04:21.927026 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:04:21.927026 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:04:21.927026 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:04:21.927026 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:04:21.927026 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:04:21.927026 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:04:21.927026 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:04:21.928056 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:04:21.928056 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:04:21.928056 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:04:21.928056 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:04:21.928056 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:04:21.928056 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:04:21.928056 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:04:21.928056 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:04:21.928056 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:04:21.928056 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:04:21.928056 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:04:21.928056 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:04:21.928056 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:04:21.928056 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:04:21.928056 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:04:21.928056 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:04:21.928895 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:04:21.928895 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:04:21.928895 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:04:21.928895 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:04:21.928895 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:04:21.928895 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:04:21.928895 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:04:21.928895 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:04:21.928895 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:04:21.928895 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:04:21.928895 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 25 03:04:21.928895 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:04:21.928895 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 624/1235] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Aug 25 03:04:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:04:21.934308 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bda948-58ce-4bf7-9bae-48b67076bdb2 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:21.980458 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:04:22.031558 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bda948-58ce-4bf7-9bae-48b67076bdb2 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:22.064607 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e92201d-cd6a-4ea9-a61d-0f9299563883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:22.065208 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e92201d-cd6a-4ea9-a61d-0f9299563883 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:22.065487 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e92201d-cd6a-4ea9-a61d-0f9299563883 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:22.065757 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e92201d-cd6a-4ea9-a61d-0f9299563883 None admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:22.066446 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e92201d-cd6a-4ea9-a61d-0f9299563883 None admin] RBAC: Adding query filter params (name=service) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:04:22.066748 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e92201d-cd6a-4ea9-a61d-0f9299563883 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:04:22.068115 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e92201d-cd6a-4ea9-a61d-0f9299563883 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:04:22.077349 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 612/1236] 10.222.0.58 () {56 vars in 1103 bytes} [Thu Aug 25 03:04:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:04:22.083380 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9997cb-57c0-45bc-9d36-fad7f49faffa None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:22.174369 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:04:22.193432 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9997cb-57c0-45bc-9d36-fad7f49faffa None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:22.226319 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac1e09b-1cdc-4e51-bbb6-de6c2837c899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:22.226949 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac1e09b-1cdc-4e51-bbb6-de6c2837c899 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:22.227231 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac1e09b-1cdc-4e51-bbb6-de6c2837c899 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:22.227503 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac1e09b-1cdc-4e51-bbb6-de6c2837c899 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:22.237142 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac1e09b-1cdc-4e51-bbb6-de6c2837c899 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:04:22.238612 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac1e09b-1cdc-4e51-bbb6-de6c2837c899 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:04:22.253223 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-cac1e09b-1cdc-4e51-bbb6-de6c2837c899 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:04:22.253223 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:04:22.253223 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:04:22.253223 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:04:22.253223 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:04:22.253223 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:04:22.253223 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:04:22.253223 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:04:22.253223 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:04:22.253223 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:04:22.253223 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:04:22.253223 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:04:22.253223 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 03:04:22.253223 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 03:04:22.253223 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 03:04:22.253223 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 03:04:22.254378 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:04:22.254378 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:04:22.254378 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:04:22.254378 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:04:22.254378 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:04:22.254378 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:04:22.254378 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:04:22.254378 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:04:22.254378 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:04:22.254378 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:04:22.254378 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:04:22.254378 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:04:22.254378 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:04:22.254378 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:04:22.254378 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:04:22.254378 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:04:22.254378 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 03:04:22.255615 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 03:04:22.255615 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:04:22.255615 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:04:22.255615 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 625/1237] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:04:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:04:22.260416 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e92201d-cd6a-4ea9-a61d-0f9299563883 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:22.359640 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e92201d-cd6a-4ea9-a61d-0f9299563883 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:22.368085 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:04:22.395751 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09800cdf-3251-4130-8dd3-576637e189bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:22.396354 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09800cdf-3251-4130-8dd3-576637e189bc None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:22.396631 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09800cdf-3251-4130-8dd3-576637e189bc None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:22.396902 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09800cdf-3251-4130-8dd3-576637e189bc None admin] PATH_INFO: `/v3/domains` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:22.397536 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09800cdf-3251-4130-8dd3-576637e189bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:04:22.397836 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09800cdf-3251-4130-8dd3-576637e189bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:04:22.399229 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09800cdf-3251-4130-8dd3-576637e189bc None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:04:22.413272 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 613/1238] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Aug 25 03:04:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:04:22.419274 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac1e09b-1cdc-4e51-bbb6-de6c2837c899 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:22.519202 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac1e09b-1cdc-4e51-bbb6-de6c2837c899 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:22.553313 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f50928-f763-4d7d-82af-d456baa2ce57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:22.554034 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f50928-f763-4d7d-82af-d456baa2ce57 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:22.554408 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f50928-f763-4d7d-82af-d456baa2ce57 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:22.554845 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f50928-f763-4d7d-82af-d456baa2ce57 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:22.558548 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:04:22.564092 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f50928-f763-4d7d-82af-d456baa2ce57 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:04:22.565647 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f50928-f763-4d7d-82af-d456baa2ce57 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:04:22.576491 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-d2f50928-f763-4d7d-82af-d456baa2ce57 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 25 03:04:22.576491 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:04:22.576491 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:04:22.576491 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:04:22.576491 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:04:22.576491 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:04:22.576491 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:04:22.576491 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:04:22.576491 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:04:22.576491 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:04:22.576491 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:04:22.576491 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:04:22.576491 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 25 03:04:22.576491 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 25 03:04:22.576491 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 25 03:04:22.576491 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 25 03:04:22.577715 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:04:22.577715 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:04:22.577715 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 25 03:04:22.577715 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 03:04:22.577715 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 25 03:04:22.577715 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 03:04:22.577715 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:04:22.577715 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:04:22.577715 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:04:22.577715 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:04:22.577715 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:04:22.577715 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:04:22.577715 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:04:22.577715 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:04:22.577715 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:04:22.577715 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:04:22.577715 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:04:22.577715 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:04:22.578875 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:04:22.578875 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:04:22.578875 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 25 03:04:22.578875 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 25 03:04:22.578875 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 25 03:04:22.578875 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 25 03:04:22.578875 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 25 03:04:22.578875 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 25 03:04:22.578875 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 25 03:04:22.578875 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:04:22.578875 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 626/1239] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:04:22 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 03:04:22.584411 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09800cdf-3251-4130-8dd3-576637e189bc None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:22.684715 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09800cdf-3251-4130-8dd3-576637e189bc None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:22.717905 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dd737d-2339-449f-92db-0a61acc41fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:22.718590 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dd737d-2339-449f-92db-0a61acc41fcf None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:22.718873 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dd737d-2339-449f-92db-0a61acc41fcf None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:22.719147 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dd737d-2339-449f-92db-0a61acc41fcf None admin] PATH_INFO: `/v3/users` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:22.719957 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dd737d-2339-449f-92db-0a61acc41fcf None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:04:22.720266 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dd737d-2339-449f-92db-0a61acc41fcf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:04:22.721691 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dd737d-2339-449f-92db-0a61acc41fcf None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:04:22.739016 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 614/1240] 10.222.0.58 () {56 vars in 1143 bytes} [Thu Aug 25 03:04:22 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:04:22.744819 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f50928-f763-4d7d-82af-d456baa2ce57 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:22.844064 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f50928-f763-4d7d-82af-d456baa2ce57 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:22.877937 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243d6eb8-93a4-4049-b6d2-2273d1b3e918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:22.878572 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243d6eb8-93a4-4049-b6d2-2273d1b3e918 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:22.878852 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243d6eb8-93a4-4049-b6d2-2273d1b3e918 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:22.879121 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243d6eb8-93a4-4049-b6d2-2273d1b3e918 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:22.938537 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:04:22.945107 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243d6eb8-93a4-4049-b6d2-2273d1b3e918 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:04:22.946586 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243d6eb8-93a4-4049-b6d2-2273d1b3e918 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:04:23.044911 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 627/1241] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Aug 25 03:04:22 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 25 03:04:23.050958 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dd737d-2339-449f-92db-0a61acc41fcf None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:23.077578 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:04:23.146070 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dd737d-2339-449f-92db-0a61acc41fcf None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:23.180255 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9854115-fec6-416b-b6f7-93fd344eb080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:23.180882 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9854115-fec6-416b-b6f7-93fd344eb080 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:23.181164 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9854115-fec6-416b-b6f7-93fd344eb080 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:23.181439 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9854115-fec6-416b-b6f7-93fd344eb080 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:23.191164 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9854115-fec6-416b-b6f7-93fd344eb080 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:04:23.192545 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9854115-fec6-416b-b6f7-93fd344eb080 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:04:23.206993 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-c9854115-fec6-416b-b6f7-93fd344eb080 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:04:23.206993 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:04:23.206993 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:04:23.206993 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:04:23.206993 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:04:23.206993 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:04:23.206993 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:04:23.206993 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:04:23.206993 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:04:23.206993 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:04:23.206993 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:04:23.206993 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:04:23.206993 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 03:04:23.206993 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 03:04:23.206993 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 03:04:23.206993 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 03:04:23.208130 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:04:23.208130 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:04:23.208130 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:04:23.208130 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:04:23.208130 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:04:23.208130 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:04:23.208130 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:04:23.208130 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:04:23.208130 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:04:23.208130 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:04:23.208130 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:04:23.208130 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:04:23.208130 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:04:23.208130 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:04:23.208130 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:04:23.208130 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:04:23.208130 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 03:04:23.209239 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 03:04:23.209239 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:04:23.209239 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:04:23.209239 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 615/1242] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:04:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 03:04:23.214801 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243d6eb8-93a4-4049-b6d2-2273d1b3e918 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:23.305441 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:04:23.311126 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243d6eb8-93a4-4049-b6d2-2273d1b3e918 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:23.344751 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6cbdcb-77c8-40ca-83f2-d33ef97aabb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:23.345354 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6cbdcb-77c8-40ca-83f2-d33ef97aabb4 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:23.345637 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6cbdcb-77c8-40ca-83f2-d33ef97aabb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:23.345913 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6cbdcb-77c8-40ca-83f2-d33ef97aabb4 None admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:23.346584 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6cbdcb-77c8-40ca-83f2-d33ef97aabb4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:04:23.346890 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6cbdcb-77c8-40ca-83f2-d33ef97aabb4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:04:23.348258 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6cbdcb-77c8-40ca-83f2-d33ef97aabb4 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:04:23.359165 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 628/1243] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Aug 25 03:04:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:04:23.364973 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9854115-fec6-416b-b6f7-93fd344eb080 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:23.440994 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:04:23.461854 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9854115-fec6-416b-b6f7-93fd344eb080 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:23.497700 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34d42c8-f752-440e-bed5-1d3422cc9599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:23.498509 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34d42c8-f752-440e-bed5-1d3422cc9599 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:23.498797 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34d42c8-f752-440e-bed5-1d3422cc9599 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:23.499073 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34d42c8-f752-440e-bed5-1d3422cc9599 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:23.505733 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34d42c8-f752-440e-bed5-1d3422cc9599 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:04:23.507207 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34d42c8-f752-440e-bed5-1d3422cc9599 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:04:23.518881 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-c34d42c8-f752-440e-bed5-1d3422cc9599 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 03:04:23.518881 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:04:23.518881 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:04:23.518881 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:04:23.518881 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:04:23.518881 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:04:23.518881 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:04:23.518881 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:04:23.518881 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:04:23.518881 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:04:23.518881 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:04:23.518881 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:04:23.518881 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 25 03:04:23.518881 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 25 03:04:23.518881 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 25 03:04:23.520265 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 25 03:04:23.520265 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:04:23.520265 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:04:23.520265 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:04:23.520265 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:04:23.520265 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:04:23.520265 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:04:23.520265 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:04:23.520265 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:04:23.520265 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:04:23.520265 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:04:23.520265 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:04:23.520265 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:04:23.520265 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:04:23.520265 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:04:23.520265 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:04:23.520265 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:04:23.521525 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 25 03:04:23.521525 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 25 03:04:23.521525 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 25 03:04:23.521525 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 25 03:04:23.521525 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 25 03:04:23.521525 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 25 03:04:23.521525 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 03:04:23.521525 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:04:23.521525 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 616/1244] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Aug 25 03:04:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:04:23.526162 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6cbdcb-77c8-40ca-83f2-d33ef97aabb4 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:23.625481 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6cbdcb-77c8-40ca-83f2-d33ef97aabb4 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:23.658393 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f43c6bb-0e71-48da-8c73-4e70ddd09680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:23.659001 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f43c6bb-0e71-48da-8c73-4e70ddd09680 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:23.659284 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f43c6bb-0e71-48da-8c73-4e70ddd09680 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:23.659559 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f43c6bb-0e71-48da-8c73-4e70ddd09680 None admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:23.660251 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f43c6bb-0e71-48da-8c73-4e70ddd09680 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:04:23.660559 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f43c6bb-0e71-48da-8c73-4e70ddd09680 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:04:23.661971 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f43c6bb-0e71-48da-8c73-4e70ddd09680 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:04:23.673623 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 629/1245] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Aug 25 03:04:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:04:23.680189 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34d42c8-f752-440e-bed5-1d3422cc9599 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:23.777878 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34d42c8-f752-440e-bed5-1d3422cc9599 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:23.812334 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4f270d-3717-4321-a3d6-fc5bb7b18b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:23.812955 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4f270d-3717-4321-a3d6-fc5bb7b18b32 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:23.813237 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4f270d-3717-4321-a3d6-fc5bb7b18b32 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:23.813512 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4f270d-3717-4321-a3d6-fc5bb7b18b32 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:23.814250 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4f270d-3717-4321-a3d6-fc5bb7b18b32 None admin] RBAC: Adding query filter params (role.id=a93ddb1fd6f6459fb46a17573b821422, scope.project.id=9fd4be43faec4e4191e0e4ec4a4588c3, user.id=2b692edd3e8c41289cc42e1afb4a6dd8) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:04:23.814590 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4f270d-3717-4321-a3d6-fc5bb7b18b32 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:04:23.816118 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4f270d-3717-4321-a3d6-fc5bb7b18b32 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:04:23.825650 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 617/1246] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Aug 25 03:04:23 2022] GET /identity/v3/role_assignments?user.id=2b692edd3e8c41289cc42e1afb4a6dd8&scope.project.id=9fd4be43faec4e4191e0e4ec4a4588c3&role.id=a93ddb1fd6f6459fb46a17573b821422 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:04:25.838667 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc02eb63-cf99-42c4-acf5-717540d8e9fa None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:25.839203 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc02eb63-cf99-42c4-acf5-717540d8e9fa None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:25.839203 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc02eb63-cf99-42c4-acf5-717540d8e9fa None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:25.840704 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 630/1247] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:04:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:04:25.846023 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d0562b-fea3-44ca-a36f-2d7291627e1e None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:25.846492 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d0562b-fea3-44ca-a36f-2d7291627e1e None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:25.846611 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d0562b-fea3-44ca-a36f-2d7291627e1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:25.852961 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:04:25.865240 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-b6d0562b-fea3-44ca-a36f-2d7291627e1e None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:04:25.866460 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-b6d0562b-fea3-44ca-a36f-2d7291627e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:04:25.931607 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 618/1248] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:04:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 03:04:26.204031 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc02eb63-cf99-42c4-acf5-717540d8e9fa None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:26.222468 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:04:26.302457 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc02eb63-cf99-42c4-acf5-717540d8e9fa None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:26.335191 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325d39b7-4442-4e0a-882a-ec976f3f1ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:26.335813 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325d39b7-4442-4e0a-882a-ec976f3f1ace None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:26.336093 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325d39b7-4442-4e0a-882a-ec976f3f1ace None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:26.336363 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325d39b7-4442-4e0a-882a-ec976f3f1ace None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:26.346745 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325d39b7-4442-4e0a-882a-ec976f3f1ace None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:04:26.351392 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-325d39b7-4442-4e0a-882a-ec976f3f1ace None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 03:04:26.351392 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:04:26.351392 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:04:26.351392 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:04:26.351392 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:04:26.351392 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:04:26.351392 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:04:26.351392 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:04:26.351392 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:04:26.351392 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:04:26.351392 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:04:26.351392 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:04:26.351392 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:04:26.351392 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:04:26.351392 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:04:26.351392 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:04:26.352302 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:04:26.352302 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:04:26.352302 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:04:26.352302 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:04:26.352302 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:04:26.352302 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:04:26.352302 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:04:26.352302 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:04:26.352302 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:04:26.352302 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:04:26.352302 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:04:26.352302 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:04:26.352302 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:04:26.352302 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:04:26.352302 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:04:26.352302 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:04:26.353145 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:04:26.353145 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:04:26.353145 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:04:26.353145 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:04:26.353145 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:04:26.353145 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:04:26.353145 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:04:26.353145 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:04:26.353145 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:04:26.353145 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:04:26.353145 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 03:04:26.353145 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:04:26.353145 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 631/1249] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Aug 25 03:04:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:04:26.358684 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d0562b-fea3-44ca-a36f-2d7291627e1e None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:26.456217 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d0562b-fea3-44ca-a36f-2d7291627e1e None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:26.485696 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:04:26.490562 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7bb04f-fb76-48aa-8825-d367663f69b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:26.491212 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7bb04f-fb76-48aa-8825-d367663f69b3 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:26.491488 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7bb04f-fb76-48aa-8825-d367663f69b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:26.491760 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7bb04f-fb76-48aa-8825-d367663f69b3 None admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:26.492421 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7bb04f-fb76-48aa-8825-d367663f69b3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:04:26.492719 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7bb04f-fb76-48aa-8825-d367663f69b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:04:26.494085 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7bb04f-fb76-48aa-8825-d367663f69b3 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:04:26.501818 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 619/1250] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Aug 25 03:04:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:04:26.507711 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325d39b7-4442-4e0a-882a-ec976f3f1ace None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:26.586359 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:04:26.608942 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325d39b7-4442-4e0a-882a-ec976f3f1ace None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:26.642593 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f903245c-bd03-4b7c-a9c8-14e1f19409a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:26.643225 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f903245c-bd03-4b7c-a9c8-14e1f19409a2 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:26.643503 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f903245c-bd03-4b7c-a9c8-14e1f19409a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:26.643775 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f903245c-bd03-4b7c-a9c8-14e1f19409a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:26.653905 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f903245c-bd03-4b7c-a9c8-14e1f19409a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:04:26.655380 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f903245c-bd03-4b7c-a9c8-14e1f19409a2 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:04:26.670106 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-f903245c-bd03-4b7c-a9c8-14e1f19409a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:04:26.670106 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:04:26.670106 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:04:26.670106 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:04:26.670106 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:04:26.670106 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:04:26.670106 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:04:26.670106 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:04:26.670106 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:04:26.670106 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:04:26.670106 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:04:26.670106 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:04:26.670106 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 03:04:26.670106 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 03:04:26.670106 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 03:04:26.670106 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 03:04:26.671308 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:04:26.671308 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:04:26.671308 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:04:26.671308 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:04:26.671308 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:04:26.671308 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:04:26.671308 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:04:26.671308 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:04:26.671308 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:04:26.671308 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:04:26.671308 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:04:26.671308 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:04:26.671308 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:04:26.671308 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:04:26.671308 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:04:26.671308 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:04:26.671308 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 03:04:26.672441 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 03:04:26.672441 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:04:26.672441 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:04:26.672441 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 632/1251] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:04:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:04:26.677254 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7bb04f-fb76-48aa-8825-d367663f69b3 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:26.774942 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7bb04f-fb76-48aa-8825-d367663f69b3 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:26.806494 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4157c8-7a11-4516-a149-ceee501f0b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:26.807094 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4157c8-7a11-4516-a149-ceee501f0b43 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:26.807904 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4157c8-7a11-4516-a149-ceee501f0b43 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:26.808175 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4157c8-7a11-4516-a149-ceee501f0b43 None admin] PATH_INFO: `/v3/domains` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:26.808811 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4157c8-7a11-4516-a149-ceee501f0b43 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:04:26.809110 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4157c8-7a11-4516-a149-ceee501f0b43 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:04:26.810500 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4157c8-7a11-4516-a149-ceee501f0b43 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:04:26.821716 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 620/1252] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Aug 25 03:04:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:04:26.828026 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f903245c-bd03-4b7c-a9c8-14e1f19409a2 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:26.926986 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f903245c-bd03-4b7c-a9c8-14e1f19409a2 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:26.953498 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:04:26.961587 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff46883-88be-4295-a5b4-6f720fb1748e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:26.962220 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff46883-88be-4295-a5b4-6f720fb1748e None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:26.962540 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff46883-88be-4295-a5b4-6f720fb1748e None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:26.962811 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff46883-88be-4295-a5b4-6f720fb1748e None admin] PATH_INFO: `/v3/users/placement` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:26.970026 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff46883-88be-4295-a5b4-6f720fb1748e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:04:26.971619 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff46883-88be-4295-a5b4-6f720fb1748e None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:04:26.982747 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-bff46883-88be-4295-a5b4-6f720fb1748e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 25 03:04:26.982747 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:04:26.982747 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:04:26.982747 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:04:26.982747 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:04:26.982747 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:04:26.982747 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:04:26.982747 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:04:26.982747 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:04:26.982747 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:04:26.982747 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:04:26.982747 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:04:26.982747 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 25 03:04:26.982747 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 25 03:04:26.982747 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 25 03:04:26.982747 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 25 03:04:26.983943 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:04:26.983943 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:04:26.983943 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 25 03:04:26.983943 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 03:04:26.983943 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 25 03:04:26.983943 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 03:04:26.983943 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:04:26.983943 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:04:26.983943 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:04:26.983943 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:04:26.983943 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:04:26.983943 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:04:26.983943 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:04:26.983943 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:04:26.983943 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:04:26.983943 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:04:26.983943 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:04:26.983943 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:04:26.985081 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:04:26.985081 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:04:26.985081 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 25 03:04:26.985081 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 25 03:04:26.985081 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 25 03:04:26.985081 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 25 03:04:26.985081 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 25 03:04:26.985081 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 25 03:04:26.985081 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 25 03:04:26.985081 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:04:26.985081 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 633/1253] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:04:26 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:04:26.989891 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4157c8-7a11-4516-a149-ceee501f0b43 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:27.052799 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:04:27.100969 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4157c8-7a11-4516-a149-ceee501f0b43 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:27.133735 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbd23f8-7124-4c3c-bca9-29d5fd75e3fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:27.134361 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbd23f8-7124-4c3c-bca9-29d5fd75e3fa None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:27.134642 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbd23f8-7124-4c3c-bca9-29d5fd75e3fa None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:27.134915 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbd23f8-7124-4c3c-bca9-29d5fd75e3fa None admin] PATH_INFO: `/v3/users` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:27.135667 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbd23f8-7124-4c3c-bca9-29d5fd75e3fa None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:04:27.135968 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbd23f8-7124-4c3c-bca9-29d5fd75e3fa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:04:27.137402 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbd23f8-7124-4c3c-bca9-29d5fd75e3fa None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:04:27.155747 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 621/1254] 10.222.0.58 () {56 vars in 1143 bytes} [Thu Aug 25 03:04:26 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:04:27.161663 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff46883-88be-4295-a5b4-6f720fb1748e None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:27.262461 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff46883-88be-4295-a5b4-6f720fb1748e None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:27.298077 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb63c2e4-5337-496d-9126-ee0021d36f7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:27.298977 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb63c2e4-5337-496d-9126-ee0021d36f7e None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:27.299257 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb63c2e4-5337-496d-9126-ee0021d36f7e None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:27.299533 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb63c2e4-5337-496d-9126-ee0021d36f7e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:27.356972 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:04:27.368896 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb63c2e4-5337-496d-9126-ee0021d36f7e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:04:27.371065 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb63c2e4-5337-496d-9126-ee0021d36f7e None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:04:27.469042 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 634/1255] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Aug 25 03:04:27 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 25 03:04:27.474995 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbd23f8-7124-4c3c-bca9-29d5fd75e3fa None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:27.577835 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbd23f8-7124-4c3c-bca9-29d5fd75e3fa None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:27.594769 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:04:27.620301 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70eb3a8-862d-4020-ad89-0f0691d8678c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:27.620866 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70eb3a8-862d-4020-ad89-0f0691d8678c None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:27.621167 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70eb3a8-862d-4020-ad89-0f0691d8678c None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:27.621419 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70eb3a8-862d-4020-ad89-0f0691d8678c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:27.634070 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70eb3a8-862d-4020-ad89-0f0691d8678c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:04:27.635607 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70eb3a8-862d-4020-ad89-0f0691d8678c None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:04:27.650375 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-c70eb3a8-862d-4020-ad89-0f0691d8678c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:04:27.650375 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:04:27.650375 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:04:27.650375 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:04:27.650375 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:04:27.650375 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:04:27.650375 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:04:27.650375 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:04:27.650375 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:04:27.650375 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:04:27.650375 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:04:27.650375 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:04:27.650375 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 03:04:27.650375 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 03:04:27.650375 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 03:04:27.650375 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 03:04:27.652011 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:04:27.652011 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:04:27.652011 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:04:27.652011 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:04:27.652011 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:04:27.652011 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:04:27.652011 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:04:27.652011 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:04:27.652011 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:04:27.652011 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:04:27.652011 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:04:27.652011 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:04:27.652011 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:04:27.652011 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:04:27.652011 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:04:27.652011 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:04:27.652011 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 03:04:27.652931 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 03:04:27.652931 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:04:27.652931 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:04:27.652931 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 622/1256] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:04:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:04:27.657812 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb63c2e4-5337-496d-9126-ee0021d36f7e None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:27.744662 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:04:27.756682 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb63c2e4-5337-496d-9126-ee0021d36f7e None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:27.790985 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd4043a-0176-45d4-863e-639ae2f0387f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:27.791596 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd4043a-0176-45d4-863e-639ae2f0387f None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:27.791876 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd4043a-0176-45d4-863e-639ae2f0387f None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:27.792151 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd4043a-0176-45d4-863e-639ae2f0387f None admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:27.792788 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd4043a-0176-45d4-863e-639ae2f0387f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:04:27.793093 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd4043a-0176-45d4-863e-639ae2f0387f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:04:27.794477 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd4043a-0176-45d4-863e-639ae2f0387f None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:04:27.805391 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 635/1257] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Aug 25 03:04:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:04:27.811221 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70eb3a8-862d-4020-ad89-0f0691d8678c None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:27.912827 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70eb3a8-862d-4020-ad89-0f0691d8678c None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:27.951059 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40646438-07d8-4775-8d28-e8456b981210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:27.951689 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40646438-07d8-4775-8d28-e8456b981210 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:27.951972 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40646438-07d8-4775-8d28-e8456b981210 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:27.952249 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40646438-07d8-4775-8d28-e8456b981210 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:27.959101 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40646438-07d8-4775-8d28-e8456b981210 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:04:27.960612 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40646438-07d8-4775-8d28-e8456b981210 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:04:27.972436 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-40646438-07d8-4775-8d28-e8456b981210 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 03:04:27.972436 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:04:27.972436 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:04:27.972436 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:04:27.972436 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:04:27.972436 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:04:27.972436 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:04:27.972436 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:04:27.972436 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:04:27.972436 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:04:27.972436 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:04:27.972436 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:04:27.972436 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 25 03:04:27.972436 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 25 03:04:27.972436 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 25 03:04:27.973548 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 25 03:04:27.973548 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:04:27.973548 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:04:27.973548 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:04:27.973548 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:04:27.973548 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:04:27.973548 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:04:27.973548 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:04:27.973548 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:04:27.973548 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:04:27.973548 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:04:27.973548 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:04:27.973548 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:04:27.973548 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:04:27.973548 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:04:27.973548 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:04:27.973548 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:04:27.974880 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 25 03:04:27.974880 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 25 03:04:27.974880 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 25 03:04:27.974880 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 25 03:04:27.974880 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 25 03:04:27.974880 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 25 03:04:27.974880 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 03:04:27.974880 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:04:27.974880 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 623/1258] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Aug 25 03:04:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:04:27.979511 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd4043a-0176-45d4-863e-639ae2f0387f None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:28.078409 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd4043a-0176-45d4-863e-639ae2f0387f None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:28.112001 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e78dc1-bd33-4db5-a8f6-ed916bc1a292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:28.112608 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e78dc1-bd33-4db5-a8f6-ed916bc1a292 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:28.112889 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e78dc1-bd33-4db5-a8f6-ed916bc1a292 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:28.113162 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e78dc1-bd33-4db5-a8f6-ed916bc1a292 None admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:28.113952 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e78dc1-bd33-4db5-a8f6-ed916bc1a292 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:04:28.114202 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e78dc1-bd33-4db5-a8f6-ed916bc1a292 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:04:28.115748 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e78dc1-bd33-4db5-a8f6-ed916bc1a292 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:04:28.128462 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 636/1259] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Aug 25 03:04:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:04:28.134787 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40646438-07d8-4775-8d28-e8456b981210 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:28.160971 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:04:28.233524 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40646438-07d8-4775-8d28-e8456b981210 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:28.266904 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fa0f0c-79ea-45dc-bcd3-6d645d655c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:28.267514 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fa0f0c-79ea-45dc-bcd3-6d645d655c72 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:28.267800 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fa0f0c-79ea-45dc-bcd3-6d645d655c72 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:28.268125 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fa0f0c-79ea-45dc-bcd3-6d645d655c72 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:28.268873 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fa0f0c-79ea-45dc-bcd3-6d645d655c72 None admin] RBAC: Adding query filter params (role.id=62f19ea272b447ccaab1576cd9bf4fd9, scope.project.id=9fd4be43faec4e4191e0e4ec4a4588c3, user.id=2b692edd3e8c41289cc42e1afb4a6dd8) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:04:28.269178 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fa0f0c-79ea-45dc-bcd3-6d645d655c72 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:04:28.270665 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fa0f0c-79ea-45dc-bcd3-6d645d655c72 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:04:28.277738 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 624/1260] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Aug 25 03:04:28 2022] GET /identity/v3/role_assignments?user.id=2b692edd3e8c41289cc42e1afb4a6dd8&scope.project.id=9fd4be43faec4e4191e0e4ec4a4588c3&role.id=62f19ea272b447ccaab1576cd9bf4fd9 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:04:30.158760 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015e5a55-f28f-420c-a919-fae6135e2684 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:30.159684 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015e5a55-f28f-420c-a919-fae6135e2684 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:30.159684 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015e5a55-f28f-420c-a919-fae6135e2684 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:30.161196 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 637/1261] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:04:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:04:30.167022 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83822975-a22a-44ad-9d2b-8cf0947b79f4 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:30.167460 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83822975-a22a-44ad-9d2b-8cf0947b79f4 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:30.167554 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83822975-a22a-44ad-9d2b-8cf0947b79f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:30.186083 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-83822975-a22a-44ad-9d2b-8cf0947b79f4 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:04:30.187304 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-83822975-a22a-44ad-9d2b-8cf0947b79f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:04:30.253188 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 625/1262] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:04:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 03:04:30.523292 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015e5a55-f28f-420c-a919-fae6135e2684 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:30.534587 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:04:30.625114 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015e5a55-f28f-420c-a919-fae6135e2684 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:30.657947 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2760bd41-adc6-44ab-b529-fde3f05eb50e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:30.658605 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2760bd41-adc6-44ab-b529-fde3f05eb50e None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:30.658889 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2760bd41-adc6-44ab-b529-fde3f05eb50e None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:30.659162 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2760bd41-adc6-44ab-b529-fde3f05eb50e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:30.670456 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2760bd41-adc6-44ab-b529-fde3f05eb50e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:04:30.675142 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-2760bd41-adc6-44ab-b529-fde3f05eb50e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 03:04:30.675142 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:04:30.675142 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:04:30.675142 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:04:30.675142 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:04:30.675142 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:04:30.675142 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:04:30.675142 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:04:30.675142 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:04:30.675142 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:04:30.675142 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:04:30.675142 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:04:30.675142 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:04:30.675142 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:04:30.675142 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:04:30.675142 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:04:30.676301 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:04:30.676301 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:04:30.676301 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:04:30.676301 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:04:30.676301 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:04:30.676301 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:04:30.676301 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:04:30.676301 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:04:30.676301 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:04:30.676301 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:04:30.676301 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:04:30.676301 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:04:30.676301 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:04:30.676301 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:04:30.676301 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:04:30.676301 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:04:30.677524 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:04:30.677524 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:04:30.677524 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:04:30.677524 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:04:30.677524 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:04:30.677524 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:04:30.677524 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:04:30.677524 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:04:30.677524 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:04:30.677524 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:04:30.677524 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 03:04:30.677524 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:04:30.677524 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 638/1263] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Aug 25 03:04:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:04:30.682454 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83822975-a22a-44ad-9d2b-8cf0947b79f4 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:30.701368 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:04:30.779953 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83822975-a22a-44ad-9d2b-8cf0947b79f4 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:30.813526 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cf043d-1e53-4b3b-aa72-28a2b85c2b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:30.814129 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cf043d-1e53-4b3b-aa72-28a2b85c2b0f None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:30.814442 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cf043d-1e53-4b3b-aa72-28a2b85c2b0f None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:30.814721 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cf043d-1e53-4b3b-aa72-28a2b85c2b0f None admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:30.815383 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cf043d-1e53-4b3b-aa72-28a2b85c2b0f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:04:30.815687 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cf043d-1e53-4b3b-aa72-28a2b85c2b0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:04:30.817099 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cf043d-1e53-4b3b-aa72-28a2b85c2b0f None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:04:30.825556 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 626/1264] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Aug 25 03:04:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:04:30.831696 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2760bd41-adc6-44ab-b529-fde3f05eb50e None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:30.909460 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:04:30.941206 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2760bd41-adc6-44ab-b529-fde3f05eb50e None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:30.974662 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9338a548-549d-4eed-9785-97568c26769f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:30.975289 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9338a548-549d-4eed-9785-97568c26769f None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:30.975678 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9338a548-549d-4eed-9785-97568c26769f None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:30.976141 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9338a548-549d-4eed-9785-97568c26769f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:30.987877 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9338a548-549d-4eed-9785-97568c26769f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:04:30.989312 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9338a548-549d-4eed-9785-97568c26769f None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:04:31.003258 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-9338a548-549d-4eed-9785-97568c26769f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:04:31.003258 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:04:31.003258 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:04:31.003258 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:04:31.003258 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:04:31.003258 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:04:31.003258 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:04:31.003258 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:04:31.003258 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:04:31.003258 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:04:31.003258 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:04:31.003258 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:04:31.003258 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 03:04:31.003258 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 03:04:31.003258 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 03:04:31.003258 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 03:04:31.008353 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:04:31.008353 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:04:31.008353 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:04:31.008353 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:04:31.008353 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:04:31.008353 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:04:31.008353 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:04:31.008353 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:04:31.008353 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:04:31.008353 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:04:31.008353 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:04:31.008353 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:04:31.008353 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:04:31.008353 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:04:31.008353 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:04:31.008353 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:04:31.008353 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 03:04:31.009561 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 03:04:31.009561 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:04:31.009561 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:04:31.009561 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 639/1265] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:04:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:04:31.010944 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cf043d-1e53-4b3b-aa72-28a2b85c2b0f None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:31.092055 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:04:31.110247 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cf043d-1e53-4b3b-aa72-28a2b85c2b0f None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:31.143022 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603e1fa3-5136-4b85-9911-52d13e48223f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:31.143617 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603e1fa3-5136-4b85-9911-52d13e48223f None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:31.143896 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603e1fa3-5136-4b85-9911-52d13e48223f None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:31.144169 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603e1fa3-5136-4b85-9911-52d13e48223f None admin] PATH_INFO: `/v3/domains` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:31.144806 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603e1fa3-5136-4b85-9911-52d13e48223f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:04:31.145107 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603e1fa3-5136-4b85-9911-52d13e48223f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:04:31.146537 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603e1fa3-5136-4b85-9911-52d13e48223f None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:04:31.157401 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 627/1266] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Aug 25 03:04:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:04:31.163095 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9338a548-549d-4eed-9785-97568c26769f None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:31.265239 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9338a548-549d-4eed-9785-97568c26769f None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:31.284787 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:04:31.298468 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11101c2-73e7-490a-9700-373db4e2c818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:31.299094 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11101c2-73e7-490a-9700-373db4e2c818 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:31.299368 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11101c2-73e7-490a-9700-373db4e2c818 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:31.299638 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11101c2-73e7-490a-9700-373db4e2c818 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:31.306832 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11101c2-73e7-490a-9700-373db4e2c818 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:04:31.308788 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11101c2-73e7-490a-9700-373db4e2c818 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:04:31.319493 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-d11101c2-73e7-490a-9700-373db4e2c818 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 25 03:04:31.319493 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:04:31.319493 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:04:31.319493 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:04:31.319493 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:04:31.319493 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:04:31.319493 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:04:31.319493 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:04:31.319493 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:04:31.319493 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:04:31.319493 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:04:31.319493 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:04:31.319493 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 25 03:04:31.319493 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 25 03:04:31.319493 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 25 03:04:31.319493 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 25 03:04:31.320617 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:04:31.320617 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:04:31.320617 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 25 03:04:31.320617 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 03:04:31.320617 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 25 03:04:31.320617 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 03:04:31.320617 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:04:31.320617 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:04:31.320617 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:04:31.320617 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:04:31.320617 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:04:31.320617 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:04:31.320617 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:04:31.320617 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:04:31.320617 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:04:31.320617 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:04:31.320617 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:04:31.320617 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:04:31.321996 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:04:31.321996 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:04:31.321996 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 25 03:04:31.321996 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 25 03:04:31.321996 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 25 03:04:31.321996 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 25 03:04:31.321996 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 25 03:04:31.321996 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 25 03:04:31.321996 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 25 03:04:31.321996 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:04:31.321996 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 640/1267] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:04:31 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:04:31.327184 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603e1fa3-5136-4b85-9911-52d13e48223f None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:31.426566 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603e1fa3-5136-4b85-9911-52d13e48223f None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:31.460151 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589a247b-8c4d-4694-b7e9-36847f118f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:31.460800 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589a247b-8c4d-4694-b7e9-36847f118f6e None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:31.461081 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589a247b-8c4d-4694-b7e9-36847f118f6e None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:31.461357 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589a247b-8c4d-4694-b7e9-36847f118f6e None admin] PATH_INFO: `/v3/users` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:31.462114 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589a247b-8c4d-4694-b7e9-36847f118f6e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:04:31.462450 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589a247b-8c4d-4694-b7e9-36847f118f6e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:04:31.463871 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589a247b-8c4d-4694-b7e9-36847f118f6e None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:04:31.473628 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:04:31.482121 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 628/1268] 10.222.0.58 () {56 vars in 1143 bytes} [Thu Aug 25 03:04:31 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:04:31.488089 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11101c2-73e7-490a-9700-373db4e2c818 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:31.588335 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11101c2-73e7-490a-9700-373db4e2c818 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:31.623064 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de4a746-1731-4aed-b380-c56cbc10e760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:31.623684 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de4a746-1731-4aed-b380-c56cbc10e760 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:31.623965 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de4a746-1731-4aed-b380-c56cbc10e760 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:31.624235 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de4a746-1731-4aed-b380-c56cbc10e760 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:31.634705 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de4a746-1731-4aed-b380-c56cbc10e760 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:04:31.636182 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de4a746-1731-4aed-b380-c56cbc10e760 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:04:31.649954 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-4de4a746-1731-4aed-b380-c56cbc10e760 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:04:31.649954 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:04:31.649954 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:04:31.649954 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:04:31.649954 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:04:31.649954 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:04:31.649954 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:04:31.649954 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:04:31.649954 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:04:31.649954 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:04:31.649954 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:04:31.649954 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:04:31.649954 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 03:04:31.649954 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 03:04:31.649954 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 03:04:31.649954 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 03:04:31.651101 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:04:31.651101 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:04:31.651101 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:04:31.651101 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:04:31.651101 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:04:31.651101 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:04:31.651101 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:04:31.651101 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:04:31.651101 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:04:31.651101 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:04:31.651101 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:04:31.651101 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:04:31.651101 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:04:31.651101 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:04:31.651101 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:04:31.651101 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:04:31.651101 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 03:04:31.652239 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 03:04:31.652239 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:04:31.652239 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:04:31.652239 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 641/1269] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:04:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:04:31.657112 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589a247b-8c4d-4694-b7e9-36847f118f6e None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:31.756011 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589a247b-8c4d-4694-b7e9-36847f118f6e None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:31.788832 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dee01a0-9d21-4662-94f0-14ecbd3754c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:31.789423 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dee01a0-9d21-4662-94f0-14ecbd3754c9 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:31.789699 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dee01a0-9d21-4662-94f0-14ecbd3754c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:31.789968 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dee01a0-9d21-4662-94f0-14ecbd3754c9 None admin] PATH_INFO: `/v3/domains` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:31.790629 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dee01a0-9d21-4662-94f0-14ecbd3754c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:04:31.790929 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dee01a0-9d21-4662-94f0-14ecbd3754c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:04:31.792794 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dee01a0-9d21-4662-94f0-14ecbd3754c9 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:04:31.803662 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 629/1270] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Aug 25 03:04:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:04:31.810210 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de4a746-1731-4aed-b380-c56cbc10e760 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:31.841935 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:04:31.908826 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de4a746-1731-4aed-b380-c56cbc10e760 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:31.941100 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142532ef-8555-485a-a400-a53cbc7328ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:31.941722 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142532ef-8555-485a-a400-a53cbc7328ff None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:31.942000 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142532ef-8555-485a-a400-a53cbc7328ff None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:31.942273 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142532ef-8555-485a-a400-a53cbc7328ff None admin] PATH_INFO: `/v3/projects/service` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:31.948636 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142532ef-8555-485a-a400-a53cbc7328ff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:04:31.950051 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142532ef-8555-485a-a400-a53cbc7328ff None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:04:31.961343 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-142532ef-8555-485a-a400-a53cbc7328ff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 03:04:31.961343 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:04:31.961343 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:04:31.961343 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:04:31.961343 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:04:31.961343 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:04:31.961343 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:04:31.961343 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:04:31.961343 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:04:31.961343 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:04:31.961343 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:04:31.961343 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:04:31.961343 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 25 03:04:31.961343 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 25 03:04:31.961343 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 25 03:04:31.966151 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 25 03:04:31.966151 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:04:31.966151 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:04:31.966151 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:04:31.966151 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:04:31.966151 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:04:31.966151 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:04:31.966151 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:04:31.966151 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:04:31.966151 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:04:31.966151 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:04:31.966151 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:04:31.966151 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:04:31.966151 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:04:31.966151 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:04:31.966151 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:04:31.966151 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:04:31.967398 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 25 03:04:31.967398 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 25 03:04:31.967398 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 25 03:04:31.967398 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 25 03:04:31.967398 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 25 03:04:31.967398 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 25 03:04:31.967398 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 03:04:31.967398 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:04:31.967398 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 642/1271] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Aug 25 03:04:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:04:31.969901 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dee01a0-9d21-4662-94f0-14ecbd3754c9 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:32.027728 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:04:32.072927 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dee01a0-9d21-4662-94f0-14ecbd3754c9 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:32.105737 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad588547-7264-444b-ac5d-44c49a89773b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:32.106363 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad588547-7264-444b-ac5d-44c49a89773b None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:32.106646 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad588547-7264-444b-ac5d-44c49a89773b None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:32.106922 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad588547-7264-444b-ac5d-44c49a89773b None admin] PATH_INFO: `/v3/projects` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:32.107616 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad588547-7264-444b-ac5d-44c49a89773b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:04:32.107920 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad588547-7264-444b-ac5d-44c49a89773b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:04:32.109486 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad588547-7264-444b-ac5d-44c49a89773b None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:04:32.121297 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 630/1272] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Aug 25 03:04:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:04:32.127529 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142532ef-8555-485a-a400-a53cbc7328ff None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:32.216731 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:04:32.225022 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142532ef-8555-485a-a400-a53cbc7328ff None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:32.257337 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7562c868-640c-47a0-9bb9-71a763c739e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:32.257989 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7562c868-640c-47a0-9bb9-71a763c739e6 None admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:32.258273 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7562c868-640c-47a0-9bb9-71a763c739e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:32.258579 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7562c868-640c-47a0-9bb9-71a763c739e6 None admin] PATH_INFO: `/v3/projects/9fd4be43faec4e4191e0e4ec4a4588c3/users/2b692edd3e8c41289cc42e1afb4a6dd8/roles/62f19ea272b447ccaab1576cd9bf4fd9` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:32.260984 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7562c868-640c-47a0-9bb9-71a763c739e6 None admin] RBAC: Authorizing `identity:create_grant(project_id=9fd4be43faec4e4191e0e4ec4a4588c3, user_id=2b692edd3e8c41289cc42e1afb4a6dd8, role_id=62f19ea272b447ccaab1576cd9bf4fd9)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:04:32.262452 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7562c868-640c-47a0-9bb9-71a763c739e6 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:04:32.349162 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 643/1273] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Aug 25 03:04:32 2022] PUT /identity/v3/projects/9fd4be43faec4e4191e0e4ec4a4588c3/users/2b692edd3e8c41289cc42e1afb4a6dd8/roles/62f19ea272b447ccaab1576cd9bf4fd9 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:04:34.219274 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a58d212-9500-40b7-adfc-809c45646b14 None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:34.219835 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a58d212-9500-40b7-adfc-809c45646b14 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:34.219835 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a58d212-9500-40b7-adfc-809c45646b14 None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:34.221203 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 631/1274] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:04:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:04:34.226479 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f529440-fd30-407d-9075-b3b2b41dbcc0 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:34.226915 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f529440-fd30-407d-9075-b3b2b41dbcc0 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:34.227021 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f529440-fd30-407d-9075-b3b2b41dbcc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:34.245092 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-0f529440-fd30-407d-9075-b3b2b41dbcc0 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:04:34.246299 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-0f529440-fd30-407d-9075-b3b2b41dbcc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:04:34.311949 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 644/1275] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:04:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 03:04:34.587611 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a58d212-9500-40b7-adfc-809c45646b14 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:34.689396 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a58d212-9500-40b7-adfc-809c45646b14 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:34.706561 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:04:34.722429 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da3357d-2157-4a93-aec0-31ce6929ca15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:34.723051 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da3357d-2157-4a93-aec0-31ce6929ca15 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:34.723327 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da3357d-2157-4a93-aec0-31ce6929ca15 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:34.723597 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da3357d-2157-4a93-aec0-31ce6929ca15 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:34.733908 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da3357d-2157-4a93-aec0-31ce6929ca15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 03:04:34.738649 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-7da3357d-2157-4a93-aec0-31ce6929ca15 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 03:04:34.738649 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:04:34.738649 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:04:34.738649 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:04:34.738649 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:04:34.738649 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:04:34.738649 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:04:34.738649 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:04:34.738649 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:04:34.738649 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:04:34.738649 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:04:34.738649 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:04:34.738649 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 03:04:34.738649 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 03:04:34.738649 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 03:04:34.738649 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 03:04:34.739611 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 03:04:34.739611 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 03:04:34.739611 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 03:04:34.739611 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 03:04:34.739611 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:04:34.739611 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:04:34.739611 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:04:34.739611 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:04:34.739611 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:04:34.739611 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:04:34.739611 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:04:34.739611 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:04:34.739611 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:04:34.739611 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:04:34.739611 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:04:34.739611 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:04:34.740499 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:04:34.740499 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:04:34.740499 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:04:34.740499 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:04:34.740499 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 03:04:34.740499 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 03:04:34.740499 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 03:04:34.740499 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 03:04:34.740499 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 03:04:34.740499 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 03:04:34.740499 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 03:04:34.740499 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:04:34.740499 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 632/1276] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Aug 25 03:04:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:04:34.746423 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f529440-fd30-407d-9075-b3b2b41dbcc0 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:34.824747 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:04:34.842708 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f529440-fd30-407d-9075-b3b2b41dbcc0 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:34.875975 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ab276a-dbc5-452a-be36-0a8f02937b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:34.876568 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ab276a-dbc5-452a-be36-0a8f02937b5f None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:34.876844 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ab276a-dbc5-452a-be36-0a8f02937b5f None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:34.877110 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ab276a-dbc5-452a-be36-0a8f02937b5f None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:34.877757 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ab276a-dbc5-452a-be36-0a8f02937b5f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:04:34.878054 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ab276a-dbc5-452a-be36-0a8f02937b5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:04:34.879468 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ab276a-dbc5-452a-be36-0a8f02937b5f None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:04:34.887246 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 645/1277] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Aug 25 03:04:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:04:34.892996 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da3357d-2157-4a93-aec0-31ce6929ca15 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:34.990858 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da3357d-2157-4a93-aec0-31ce6929ca15 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:35.023734 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ec42bf-5480-46d5-b744-a5b396b8da8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:35.024362 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ec42bf-5480-46d5-b744-a5b396b8da8a None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:35.024646 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ec42bf-5480-46d5-b744-a5b396b8da8a None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:35.024921 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ec42bf-5480-46d5-b744-a5b396b8da8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:35.035241 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ec42bf-5480-46d5-b744-a5b396b8da8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:04:35.036691 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ec42bf-5480-46d5-b744-a5b396b8da8a None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:04:35.051605 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-77ec42bf-5480-46d5-b744-a5b396b8da8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:04:35.051605 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:04:35.051605 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:04:35.051605 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:04:35.051605 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:04:35.051605 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:04:35.051605 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:04:35.051605 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:04:35.051605 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:04:35.051605 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:04:35.051605 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:04:35.051605 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:04:35.051605 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 03:04:35.051605 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 03:04:35.051605 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 03:04:35.051605 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 03:04:35.057607 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:04:35.057607 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:04:35.057607 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:04:35.057607 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:04:35.057607 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:04:35.057607 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:04:35.057607 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:04:35.057607 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:04:35.057607 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:04:35.057607 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:04:35.057607 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:04:35.057607 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:04:35.057607 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:04:35.057607 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:04:35.057607 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:04:35.057607 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:04:35.057607 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 03:04:35.058739 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 03:04:35.058739 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:04:35.058739 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:04:35.058739 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 633/1278] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:04:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:04:35.060090 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ab276a-dbc5-452a-be36-0a8f02937b5f None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:35.159483 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ab276a-dbc5-452a-be36-0a8f02937b5f None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:35.192302 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286c8f93-7108-4457-98ec-d71c0d240b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:35.192906 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286c8f93-7108-4457-98ec-d71c0d240b16 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:35.193190 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286c8f93-7108-4457-98ec-d71c0d240b16 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:35.193467 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286c8f93-7108-4457-98ec-d71c0d240b16 None admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:35.194105 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286c8f93-7108-4457-98ec-d71c0d240b16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:04:35.194445 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286c8f93-7108-4457-98ec-d71c0d240b16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:04:35.195801 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286c8f93-7108-4457-98ec-d71c0d240b16 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:04:35.207413 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 646/1279] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Aug 25 03:04:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:04:35.213157 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ec42bf-5480-46d5-b744-a5b396b8da8a None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:35.243782 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:04:35.310055 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ec42bf-5480-46d5-b744-a5b396b8da8a None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:35.343099 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ea99ac-e38a-4cc2-ab43-c0491007f44b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:35.343700 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ea99ac-e38a-4cc2-ab43-c0491007f44b None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:35.343978 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ea99ac-e38a-4cc2-ab43-c0491007f44b None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:35.344243 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ea99ac-e38a-4cc2-ab43-c0491007f44b None admin] PATH_INFO: `/v3/users/placement` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:35.351120 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ea99ac-e38a-4cc2-ab43-c0491007f44b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:04:35.352581 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ea99ac-e38a-4cc2-ab43-c0491007f44b None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:04:35.364128 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-d1ea99ac-e38a-4cc2-ab43-c0491007f44b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 25 03:04:35.364128 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:04:35.364128 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:04:35.364128 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:04:35.364128 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:04:35.364128 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:04:35.364128 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:04:35.364128 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:04:35.364128 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:04:35.364128 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:04:35.364128 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:04:35.364128 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:04:35.364128 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 25 03:04:35.364128 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 25 03:04:35.364128 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 25 03:04:35.364128 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 25 03:04:35.365295 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:04:35.365295 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:04:35.365295 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 25 03:04:35.365295 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 03:04:35.365295 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 25 03:04:35.365295 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 03:04:35.365295 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:04:35.365295 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:04:35.365295 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:04:35.365295 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:04:35.365295 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:04:35.365295 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:04:35.365295 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:04:35.365295 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:04:35.365295 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:04:35.365295 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:04:35.365295 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:04:35.365295 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:04:35.366667 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:04:35.366667 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:04:35.366667 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 25 03:04:35.366667 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 25 03:04:35.366667 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 25 03:04:35.366667 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 25 03:04:35.366667 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 25 03:04:35.366667 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 25 03:04:35.366667 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 25 03:04:35.366667 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:04:35.366667 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 634/1280] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:04:35 2022] GET /identity/v3/users/placement => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:04:35.371282 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286c8f93-7108-4457-98ec-d71c0d240b16 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:35.375220 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:04:35.468552 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286c8f93-7108-4457-98ec-d71c0d240b16 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:35.503019 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b25617-583e-488c-bbf0-d7d466e985fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:35.503616 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b25617-583e-488c-bbf0-d7d466e985fe None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:35.503894 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b25617-583e-488c-bbf0-d7d466e985fe None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:35.504165 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b25617-583e-488c-bbf0-d7d466e985fe None admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:35.504877 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b25617-583e-488c-bbf0-d7d466e985fe None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:04:35.505208 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b25617-583e-488c-bbf0-d7d466e985fe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:04:35.506636 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b25617-583e-488c-bbf0-d7d466e985fe None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:04:35.525742 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 647/1281] 10.222.0.58 () {56 vars in 1143 bytes} [Thu Aug 25 03:04:35 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:04:35.532838 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ea99ac-e38a-4cc2-ab43-c0491007f44b None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:35.624820 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:04:35.634129 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ea99ac-e38a-4cc2-ab43-c0491007f44b None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:35.666998 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e51fd4-89dd-49ab-b02e-c05b0db2e867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:35.667612 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e51fd4-89dd-49ab-b02e-c05b0db2e867 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:35.667893 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e51fd4-89dd-49ab-b02e-c05b0db2e867 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:35.668160 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e51fd4-89dd-49ab-b02e-c05b0db2e867 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:35.732514 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e51fd4-89dd-49ab-b02e-c05b0db2e867 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:04:35.733878 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e51fd4-89dd-49ab-b02e-c05b0db2e867 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:04:35.828749 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:04:35.839119 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 635/1282] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Aug 25 03:04:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 25 03:04:35.845637 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b25617-583e-488c-bbf0-d7d466e985fe None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:35.908037 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:04:35.944145 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b25617-583e-488c-bbf0-d7d466e985fe None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:35.976718 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c99f80-a5a7-4814-9fba-01cfd7a53e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:35.977350 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c99f80-a5a7-4814-9fba-01cfd7a53e14 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:35.977630 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c99f80-a5a7-4814-9fba-01cfd7a53e14 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:35.977904 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c99f80-a5a7-4814-9fba-01cfd7a53e14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:35.988219 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c99f80-a5a7-4814-9fba-01cfd7a53e14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:04:35.989586 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c99f80-a5a7-4814-9fba-01cfd7a53e14 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:04:36.003559 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-a0c99f80-a5a7-4814-9fba-01cfd7a53e14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:04:36.003559 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:04:36.003559 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:04:36.003559 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:04:36.003559 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:04:36.003559 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:04:36.003559 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:04:36.003559 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:04:36.003559 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:04:36.003559 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:04:36.003559 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:04:36.003559 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:04:36.003559 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 03:04:36.003559 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 03:04:36.003559 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 03:04:36.003559 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 03:04:36.004908 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:04:36.004908 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:04:36.004908 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:04:36.004908 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:04:36.004908 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:04:36.004908 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:04:36.004908 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:04:36.004908 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:04:36.004908 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:04:36.004908 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:04:36.004908 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:04:36.004908 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:04:36.004908 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:04:36.004908 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:04:36.004908 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:04:36.004908 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:04:36.004908 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 03:04:36.005945 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 03:04:36.005945 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 03:04:36.005945 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:04:36.005945 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 648/1283] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:04:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:04:36.010440 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e51fd4-89dd-49ab-b02e-c05b0db2e867 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:36.109377 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e51fd4-89dd-49ab-b02e-c05b0db2e867 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:36.146710 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d27c221-5a7f-47f7-b034-339106d69244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:36.147318 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d27c221-5a7f-47f7-b034-339106d69244 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:36.147596 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d27c221-5a7f-47f7-b034-339106d69244 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:36.147871 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d27c221-5a7f-47f7-b034-339106d69244 None admin] PATH_INFO: `/v3/domains` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:36.148511 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d27c221-5a7f-47f7-b034-339106d69244 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:04:36.148813 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d27c221-5a7f-47f7-b034-339106d69244 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:04:36.150178 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d27c221-5a7f-47f7-b034-339106d69244 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:04:36.161775 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 636/1284] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Aug 25 03:04:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:04:36.167789 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c99f80-a5a7-4814-9fba-01cfd7a53e14 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:36.267780 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c99f80-a5a7-4814-9fba-01cfd7a53e14 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:36.280787 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:04:36.300347 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a31a734-baf5-4ee2-ba6e-389c882c5cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:36.300980 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a31a734-baf5-4ee2-ba6e-389c882c5cfe None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:36.301262 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a31a734-baf5-4ee2-ba6e-389c882c5cfe None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:36.301535 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a31a734-baf5-4ee2-ba6e-389c882c5cfe None admin] PATH_INFO: `/v3/projects/service` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:36.308978 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a31a734-baf5-4ee2-ba6e-389c882c5cfe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:04:36.310470 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a31a734-baf5-4ee2-ba6e-389c882c5cfe None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:04:36.321361 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-0a31a734-baf5-4ee2-ba6e-389c882c5cfe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 03:04:36.321361 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:04:36.321361 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:04:36.321361 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:04:36.321361 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:04:36.321361 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:04:36.321361 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:04:36.321361 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:04:36.321361 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:04:36.321361 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:04:36.321361 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:04:36.321361 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:04:36.321361 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 25 03:04:36.321361 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 25 03:04:36.321361 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 25 03:04:36.322477 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 25 03:04:36.322477 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:04:36.322477 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:04:36.322477 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:04:36.322477 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:04:36.322477 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:04:36.322477 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:04:36.322477 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:04:36.322477 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Aug 25 03:04:36.322477 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:04:36.322477 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:04:36.322477 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:04:36.322477 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:04:36.322477 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:04:36.322477 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:04:36.322477 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:04:36.322477 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:04:36.323408 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 25 03:04:36.323408 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 25 03:04:36.323408 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 25 03:04:36.323408 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 25 03:04:36.323408 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 25 03:04:36.323408 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 25 03:04:36.323408 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 03:04:36.323408 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Aug 25 03:04:36.323408 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 649/1285] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Aug 25 03:04:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:04:36.329055 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d27c221-5a7f-47f7-b034-339106d69244 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:36.379582 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:04:36.433340 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d27c221-5a7f-47f7-b034-339106d69244 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:36.469456 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f1bf86-ceba-499f-995c-3ddd6a4009ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:36.470203 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f1bf86-ceba-499f-995c-3ddd6a4009ca None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:36.470611 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f1bf86-ceba-499f-995c-3ddd6a4009ca None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:36.470895 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f1bf86-ceba-499f-995c-3ddd6a4009ca None admin] PATH_INFO: `/v3/projects` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:36.471614 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f1bf86-ceba-499f-995c-3ddd6a4009ca None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:04:36.471923 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f1bf86-ceba-499f-995c-3ddd6a4009ca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:04:36.473398 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f1bf86-ceba-499f-995c-3ddd6a4009ca None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:04:36.487036 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 637/1286] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Aug 25 03:04:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:04:36.495910 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a31a734-baf5-4ee2-ba6e-389c882c5cfe None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:36.602513 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a31a734-baf5-4ee2-ba6e-389c882c5cfe None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:36.640194 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571da2db-0714-4302-8ab1-e678e8322a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:36.640822 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571da2db-0714-4302-8ab1-e678e8322a6d None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:36.649467 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571da2db-0714-4302-8ab1-e678e8322a6d None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:36.649467 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571da2db-0714-4302-8ab1-e678e8322a6d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:36.649467 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571da2db-0714-4302-8ab1-e678e8322a6d None admin] RBAC: Adding query filter params (role.id=62f19ea272b447ccaab1576cd9bf4fd9, scope.project.id=9fd4be43faec4e4191e0e4ec4a4588c3, user.id=2b692edd3e8c41289cc42e1afb4a6dd8) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:04:36.649467 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571da2db-0714-4302-8ab1-e678e8322a6d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:04:36.649467 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571da2db-0714-4302-8ab1-e678e8322a6d None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:04:36.650266 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 650/1287] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Aug 25 03:04:36 2022] GET /identity/v3/role_assignments?user.id=2b692edd3e8c41289cc42e1afb4a6dd8&scope.project.id=9fd4be43faec4e4191e0e4ec4a4588c3&role.id=62f19ea272b447ccaab1576cd9bf4fd9 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:04:38.591807 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a2de12-aa84-4d58-bd5b-b3bcce64ee87 None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:38.592350 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a2de12-aa84-4d58-bd5b-b3bcce64ee87 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:38.592454 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a2de12-aa84-4d58-bd5b-b3bcce64ee87 None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:38.593724 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 638/1288] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:04:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:04:38.599058 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79083e7a-fe87-42bd-9bee-315c27e1611a None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:38.599498 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79083e7a-fe87-42bd-9bee-315c27e1611a None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:38.599609 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79083e7a-fe87-42bd-9bee-315c27e1611a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:38.617703 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-79083e7a-fe87-42bd-9bee-315c27e1611a None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:04:38.618892 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-79083e7a-fe87-42bd-9bee-315c27e1611a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:04:38.648285 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:04:38.685454 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 651/1289] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:04:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 03:04:38.952741 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a2de12-aa84-4d58-bd5b-b3bcce64ee87 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:39.056641 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a2de12-aa84-4d58-bd5b-b3bcce64ee87 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:39.069276 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:04:39.093771 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e901a7-4bcc-4c6a-aafb-7c8ed6efcda1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:39.094569 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e901a7-4bcc-4c6a-aafb-7c8ed6efcda1 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:39.094853 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e901a7-4bcc-4c6a-aafb-7c8ed6efcda1 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:39.095127 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e901a7-4bcc-4c6a-aafb-7c8ed6efcda1 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:39.095715 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e901a7-4bcc-4c6a-aafb-7c8ed6efcda1 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:04:39.097075 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e901a7-4bcc-4c6a-aafb-7c8ed6efcda1 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:04:39.106736 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-a2e901a7-4bcc-4c6a-aafb-7c8ed6efcda1 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 25 03:04:39.106736 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:04:39.106736 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 25 03:04:39.106736 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 25 03:04:39.106736 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 25 03:04:39.106736 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 25 03:04:39.106736 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 25 03:04:39.106736 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 03:04:39.106736 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 25 03:04:39.106736 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:04:39.106736 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 25 03:04:39.106736 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:04:39.106736 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:04:39.106736 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:04:39.106736 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:04:39.106736 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:04:39.106736 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:04:39.107836 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:04:39.107836 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:04:39.107836 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:04:39.107836 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:04:39.107836 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:04:39.107836 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:04:39.107836 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 25 03:04:39.107836 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 25 03:04:39.107836 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 25 03:04:39.107836 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 25 03:04:39.107836 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:04:39.107836 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:04:39.107836 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:04:39.107836 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:04:39.107836 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:04:39.107836 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:04:39.108718 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:04:39.108718 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:04:39.108718 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:04:39.108718 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:04:39.108718 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:04:39.108718 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:04:39.108718 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:04:39.108718 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:04:39.108718 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:04:39.108718 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:04:39.108718 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 25 03:04:39.108718 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 03:04:39.108718 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 25 03:04:39.108718 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:04:39.108718 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 639/1290] 10.222.0.58 () {56 vars in 1117 bytes} [Thu Aug 25 03:04:38 2022] GET /identity/v3/services/placement => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:04:39.114033 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79083e7a-fe87-42bd-9bee-315c27e1611a None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:39.214315 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79083e7a-fe87-42bd-9bee-315c27e1611a None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:39.253428 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef10452-092f-4892-a8a5-ae5ba5f0ab3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:39.254043 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef10452-092f-4892-a8a5-ae5ba5f0ab3b None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:39.254437 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef10452-092f-4892-a8a5-ae5ba5f0ab3b None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:39.254710 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef10452-092f-4892-a8a5-ae5ba5f0ab3b None admin] PATH_INFO: `/v3/services` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:39.255336 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef10452-092f-4892-a8a5-ae5ba5f0ab3b None admin] RBAC: Adding query filter params (name=placement) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:04:39.255635 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef10452-092f-4892-a8a5-ae5ba5f0ab3b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:04:39.256994 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef10452-092f-4892-a8a5-ae5ba5f0ab3b None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:04:39.262752 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 652/1291] 10.222.0.58 () {56 vars in 1116 bytes} [Thu Aug 25 03:04:39 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:04:39.268570 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e901a7-4bcc-4c6a-aafb-7c8ed6efcda1 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:39.366368 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e901a7-4bcc-4c6a-aafb-7c8ed6efcda1 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:39.398798 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82f5a86-c61b-4c6e-8850-5e5e478636d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:39.399415 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82f5a86-c61b-4c6e-8850-5e5e478636d2 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:39.399694 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82f5a86-c61b-4c6e-8850-5e5e478636d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:39.399961 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82f5a86-c61b-4c6e-8850-5e5e478636d2 None admin] PATH_INFO: `/v3/services` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:39.400575 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82f5a86-c61b-4c6e-8850-5e5e478636d2 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:04:39.400872 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82f5a86-c61b-4c6e-8850-5e5e478636d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:04:39.402299 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82f5a86-c61b-4c6e-8850-5e5e478636d2 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:04:39.406583 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 640/1292] 10.222.0.58 () {56 vars in 1116 bytes} [Thu Aug 25 03:04:39 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:04:41.297431 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937ac308-38c7-4d4a-aa17-10c0329add11 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:41.297981 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937ac308-38c7-4d4a-aa17-10c0329add11 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:41.297981 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937ac308-38c7-4d4a-aa17-10c0329add11 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:41.299367 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 653/1293] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:04:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:04:41.304735 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e41e57-3bde-45e6-a9a0-1cd3386c4fbb None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:41.305142 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e41e57-3bde-45e6-a9a0-1cd3386c4fbb None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:41.305276 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e41e57-3bde-45e6-a9a0-1cd3386c4fbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:41.323781 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-b7e41e57-3bde-45e6-a9a0-1cd3386c4fbb None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:04:41.326560 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-b7e41e57-3bde-45e6-a9a0-1cd3386c4fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:04:41.354407 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:04:41.390213 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 641/1294] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:04:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 03:04:41.662578 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937ac308-38c7-4d4a-aa17-10c0329add11 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:41.693756 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:04:41.761234 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937ac308-38c7-4d4a-aa17-10c0329add11 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:41.793278 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753ea037-85e2-4a6b-90cc-ca009ded0dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:41.794429 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753ea037-85e2-4a6b-90cc-ca009ded0dc0 None admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:41.794713 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753ea037-85e2-4a6b-90cc-ca009ded0dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:41.794986 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753ea037-85e2-4a6b-90cc-ca009ded0dc0 None admin] PATH_INFO: `/v3/services` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:41.795630 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753ea037-85e2-4a6b-90cc-ca009ded0dc0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:04:41.796979 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753ea037-85e2-4a6b-90cc-ca009ded0dc0 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:04:41.868653 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 654/1295] 10.222.0.58 () {60 vars in 1141 bytes} [Thu Aug 25 03:04:41 2022] POST /identity/v3/services => generated 246 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:04:43.765678 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0382d3-0cd3-4639-8286-79d4e4c3f021 None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:43.766243 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0382d3-0cd3-4639-8286-79d4e4c3f021 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:43.766243 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0382d3-0cd3-4639-8286-79d4e4c3f021 None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:43.767600 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 642/1296] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:04:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:04:43.773019 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34883ba6-eb45-48bc-8b70-5e0d16922ec3 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:43.773421 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34883ba6-eb45-48bc-8b70-5e0d16922ec3 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:43.773569 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34883ba6-eb45-48bc-8b70-5e0d16922ec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:43.791706 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-34883ba6-eb45-48bc-8b70-5e0d16922ec3 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:04:43.792920 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-34883ba6-eb45-48bc-8b70-5e0d16922ec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:04:43.865622 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 655/1297] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:04:43 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 03:04:44.133486 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0382d3-0cd3-4639-8286-79d4e4c3f021 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:44.233161 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0382d3-0cd3-4639-8286-79d4e4c3f021 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:44.266223 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35edf819-3996-4040-acad-a430e530cfe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:44.266858 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35edf819-3996-4040-acad-a430e530cfe3 None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:44.267137 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35edf819-3996-4040-acad-a430e530cfe3 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:44.267409 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35edf819-3996-4040-acad-a430e530cfe3 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:44.267993 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35edf819-3996-4040-acad-a430e530cfe3 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:04:44.269358 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35edf819-3996-4040-acad-a430e530cfe3 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:04:44.277811 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-35edf819-3996-4040-acad-a430e530cfe3 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 25 03:04:44.277811 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:04:44.277811 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 25 03:04:44.277811 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 25 03:04:44.277811 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 25 03:04:44.277811 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 25 03:04:44.277811 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 25 03:04:44.277811 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 03:04:44.277811 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 25 03:04:44.277811 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:04:44.277811 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 25 03:04:44.277811 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:04:44.277811 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:04:44.277811 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:04:44.277811 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:04:44.277811 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:04:44.277811 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:04:44.278957 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:04:44.278957 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:04:44.278957 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:04:44.278957 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:04:44.278957 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:04:44.278957 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:04:44.278957 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 25 03:04:44.278957 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 25 03:04:44.278957 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 25 03:04:44.278957 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 25 03:04:44.278957 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:04:44.278957 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:04:44.278957 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:04:44.278957 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:04:44.278957 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:04:44.278957 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:04:44.279792 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:04:44.279792 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:04:44.279792 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:04:44.279792 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:04:44.279792 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:04:44.279792 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:04:44.279792 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:04:44.279792 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:04:44.279792 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:04:44.279792 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:04:44.279792 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 25 03:04:44.279792 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 03:04:44.279792 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 25 03:04:44.279792 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:04:44.279792 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 643/1298] 10.222.0.58 () {56 vars in 1117 bytes} [Thu Aug 25 03:04:44 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 03:04:44.285080 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34883ba6-eb45-48bc-8b70-5e0d16922ec3 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:44.303257 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:04:44.382264 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34883ba6-eb45-48bc-8b70-5e0d16922ec3 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:44.415109 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a29efaa-99ff-4f89-aeb0-e6ec41a3383c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:44.415715 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a29efaa-99ff-4f89-aeb0-e6ec41a3383c None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:44.415997 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a29efaa-99ff-4f89-aeb0-e6ec41a3383c None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:44.416271 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a29efaa-99ff-4f89-aeb0-e6ec41a3383c None admin] PATH_INFO: `/v3/services` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:44.416973 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a29efaa-99ff-4f89-aeb0-e6ec41a3383c None admin] RBAC: Adding query filter params (name=placement) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:04:44.417274 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a29efaa-99ff-4f89-aeb0-e6ec41a3383c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:04:44.418732 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a29efaa-99ff-4f89-aeb0-e6ec41a3383c None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:04:44.423707 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 656/1299] 10.222.0.58 () {56 vars in 1116 bytes} [Thu Aug 25 03:04:44 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:04:44.429906 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35edf819-3996-4040-acad-a430e530cfe3 None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:44.462445 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:04:44.529481 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35edf819-3996-4040-acad-a430e530cfe3 None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:44.562267 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8292a9-c797-42ad-bfbe-8f6d114fa29c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:44.562905 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8292a9-c797-42ad-bfbe-8f6d114fa29c None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:44.563188 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8292a9-c797-42ad-bfbe-8f6d114fa29c None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:44.563464 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8292a9-c797-42ad-bfbe-8f6d114fa29c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:44.564162 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8292a9-c797-42ad-bfbe-8f6d114fa29c None admin] RBAC: Adding query filter params (interface=public, service_id=49c0ae2f38ff406295312b16e13f8587, region_id=RegionOne) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:04:44.564470 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8292a9-c797-42ad-bfbe-8f6d114fa29c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:04:44.565840 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8292a9-c797-42ad-bfbe-8f6d114fa29c None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:04:44.570721 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 644/1300] 10.222.0.58 () {56 vars in 1251 bytes} [Thu Aug 25 03:04:44 2022] GET /identity/v3/endpoints?service_id=49c0ae2f38ff406295312b16e13f8587&interface=public®ion_id=RegionOne => generated 194 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:04:44.577030 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a29efaa-99ff-4f89-aeb0-e6ec41a3383c None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:44.659373 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:04:44.676613 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a29efaa-99ff-4f89-aeb0-e6ec41a3383c None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:44.709726 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4107587c-5088-4ae7-a583-ad77f69de2b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:44.710369 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4107587c-5088-4ae7-a583-ad77f69de2b8 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:44.710654 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4107587c-5088-4ae7-a583-ad77f69de2b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:44.710931 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4107587c-5088-4ae7-a583-ad77f69de2b8 None admin] PATH_INFO: `/v3/services` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:44.711541 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4107587c-5088-4ae7-a583-ad77f69de2b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:04:44.713344 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4107587c-5088-4ae7-a583-ad77f69de2b8 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:04:44.721533 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 657/1301] 10.222.0.58 () {56 vars in 1087 bytes} [Thu Aug 25 03:04:44 2022] GET /identity/v3/services => generated 2191 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:04:46.648563 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e1241c-7945-4185-98ad-37b6cef769fe None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:46.648993 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e1241c-7945-4185-98ad-37b6cef769fe None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:46.649120 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e1241c-7945-4185-98ad-37b6cef769fe None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:46.650621 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 645/1302] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:04:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:04:46.655803 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7de97f0-773a-4590-9c1d-bedfc3b2e1d3 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:46.656177 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7de97f0-773a-4590-9c1d-bedfc3b2e1d3 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:46.656362 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7de97f0-773a-4590-9c1d-bedfc3b2e1d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:46.673923 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-d7de97f0-773a-4590-9c1d-bedfc3b2e1d3 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:04:46.675154 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-d7de97f0-773a-4590-9c1d-bedfc3b2e1d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:04:46.740075 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 658/1303] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:04:46 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 03:04:47.008604 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e1241c-7945-4185-98ad-37b6cef769fe None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:47.102449 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:04:47.109052 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e1241c-7945-4185-98ad-37b6cef769fe None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:47.141196 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edd009d-d4fa-464f-b391-3d905bd5473b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:47.141845 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edd009d-d4fa-464f-b391-3d905bd5473b None admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:47.142120 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edd009d-d4fa-464f-b391-3d905bd5473b None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:47.142419 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edd009d-d4fa-464f-b391-3d905bd5473b None admin] PATH_INFO: `/v3/services/placement` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:47.143000 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edd009d-d4fa-464f-b391-3d905bd5473b None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:04:47.144365 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edd009d-d4fa-464f-b391-3d905bd5473b None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:04:47.153439 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application [None req-3edd009d-d4fa-464f-b391-3d905bd5473b None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 25 03:04:47.153439 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:04:47.153439 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 25 03:04:47.153439 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 25 03:04:47.153439 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 25 03:04:47.153439 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 25 03:04:47.153439 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 25 03:04:47.153439 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 03:04:47.153439 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 25 03:04:47.153439 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:04:47.153439 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 25 03:04:47.153439 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:04:47.153439 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 03:04:47.153439 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 03:04:47.153439 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 03:04:47.153439 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 03:04:47.153439 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 03:04:47.154703 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 03:04:47.154703 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 03:04:47.154703 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 03:04:47.154703 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 03:04:47.154703 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 03:04:47.154703 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 03:04:47.154703 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 25 03:04:47.154703 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 25 03:04:47.154703 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 25 03:04:47.154703 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 25 03:04:47.154703 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 03:04:47.154703 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 03:04:47.154703 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 03:04:47.154703 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 03:04:47.154703 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 03:04:47.154703 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 03:04:47.155740 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 03:04:47.155740 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application with Lock( Aug 25 03:04:47.155740 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 03:04:47.155740 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self._enter() Aug 25 03:04:47.155740 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 03:04:47.155740 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 03:04:47.155740 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 03:04:47.155740 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application return self.creator() Aug 25 03:04:47.155740 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 03:04:47.155740 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application created_value = creator( Aug 25 03:04:47.155740 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 25 03:04:47.155740 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 03:04:47.155740 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 25 03:04:47.155740 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: ERROR keystone.server.flask.application Aug 25 03:04:47.155740 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 646/1304] 10.222.0.58 () {56 vars in 1117 bytes} [Thu Aug 25 03:04:47 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 03:04:47.160781 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7de97f0-773a-4590-9c1d-bedfc3b2e1d3 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:47.235482 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:04:47.257221 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7de97f0-773a-4590-9c1d-bedfc3b2e1d3 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:47.289783 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3627d2be-091c-4fea-93b3-46653bba404c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:47.290630 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3627d2be-091c-4fea-93b3-46653bba404c None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:47.290915 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3627d2be-091c-4fea-93b3-46653bba404c None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:47.291191 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3627d2be-091c-4fea-93b3-46653bba404c None admin] PATH_INFO: `/v3/services` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:47.291823 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3627d2be-091c-4fea-93b3-46653bba404c None admin] RBAC: Adding query filter params (name=placement) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:04:47.292127 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3627d2be-091c-4fea-93b3-46653bba404c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:04:47.293558 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3627d2be-091c-4fea-93b3-46653bba404c None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:04:47.298606 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 659/1305] 10.222.0.58 () {56 vars in 1116 bytes} [Thu Aug 25 03:04:47 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:04:47.304543 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edd009d-d4fa-464f-b391-3d905bd5473b None admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:47.402558 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edd009d-d4fa-464f-b391-3d905bd5473b None admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:47.434760 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1942ce9-af6a-4e2b-8742-6c3cdecfd876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:47.435356 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1942ce9-af6a-4e2b-8742-6c3cdecfd876 None admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:47.435634 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1942ce9-af6a-4e2b-8742-6c3cdecfd876 None admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:47.435904 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1942ce9-af6a-4e2b-8742-6c3cdecfd876 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:47.436606 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1942ce9-af6a-4e2b-8742-6c3cdecfd876 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:04:47.437961 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1942ce9-af6a-4e2b-8742-6c3cdecfd876 None admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:04:47.542799 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 647/1306] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:04:47 2022] POST /identity/v3/endpoints => generated 331 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:04:57.320077 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=30 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:04:57.538217 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:10:47.498782 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f952419-c070-4499-9be5-ff11be948fe3 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:10:47.499848 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f952419-c070-4499-9be5-ff11be948fe3 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:10:47.500358 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f952419-c070-4499-9be5-ff11be948fe3 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:10:47.507912 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ab6389-7d45-484c-ba2b-d65738e27845 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:10:47.508797 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ab6389-7d45-484c-ba2b-d65738e27845 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:10:47.509322 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ab6389-7d45-484c-ba2b-d65738e27845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:10:47.514551 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 660/1307] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:10:47 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:10:47.589173 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-58ab6389-7d45-484c-ba2b-d65738e27845 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:10:47.590791 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-58ab6389-7d45-484c-ba2b-d65738e27845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:10:47.722808 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 648/1308] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:10:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:10:48.030978 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0328dadb-0931-494d-948c-9391117835a1 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:10:48.031541 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0328dadb-0931-494d-948c-9391117835a1 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:10:48.032025 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0328dadb-0931-494d-948c-9391117835a1 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:10:48.033580 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 661/1309] 10.222.0.58 () {54 vars in 965 bytes} [Thu Aug 25 03:10:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:10:48.054234 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bce6ba4-96bc-4a2f-b29d-54eda311a719 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:10:48.054234 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bce6ba4-96bc-4a2f-b29d-54eda311a719 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:10:48.054234 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bce6ba4-96bc-4a2f-b29d-54eda311a719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:10:48.174091 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-8bce6ba4-96bc-4a2f-b29d-54eda311a719 None None] MFA Rules not processed for user `a0a1a7f091774842907fd08de15e8058`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:10:48.175435 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-8bce6ba4-96bc-4a2f-b29d-54eda311a719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:10:48.200862 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:10:48.261382 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 649/1310] 10.222.0.58 () {58 vars in 1062 bytes} [Thu Aug 25 03:10:48 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:10:48.270336 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0328dadb-0931-494d-948c-9391117835a1 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:10:48.273678 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-0328dadb-0931-494d-948c-9391117835a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:10:48.516432 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0328dadb-0931-494d-948c-9391117835a1 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:10:48.626146 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d195ef5b-bffa-4492-8bf1-23e74fad604f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:10:48.626703 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d195ef5b-bffa-4492-8bf1-23e74fad604f service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:10:48.626980 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d195ef5b-bffa-4492-8bf1-23e74fad604f service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:10:48.627304 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d195ef5b-bffa-4492-8bf1-23e74fad604f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:10:48.629273 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-d195ef5b-bffa-4492-8bf1-23e74fad604f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:10:48.700059 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:10:48.774333 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d195ef5b-bffa-4492-8bf1-23e74fad604f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:10:48.775756 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d195ef5b-bffa-4492-8bf1-23e74fad604f service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:10:48.796878 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 662/1311] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:10:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 527 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:10:51.044386 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74565dd5-00ed-4fd8-a781-3f48d44102a0 None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:10:51.044911 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74565dd5-00ed-4fd8-a781-3f48d44102a0 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:10:51.045031 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74565dd5-00ed-4fd8-a781-3f48d44102a0 None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:10:51.046340 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 650/1312] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:10:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:10:51.051794 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e733a3-a123-4482-b71c-2932f68a7bcb None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:10:51.052252 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e733a3-a123-4482-b71c-2932f68a7bcb None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:10:51.052349 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e733a3-a123-4482-b71c-2932f68a7bcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:10:51.131330 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-59e733a3-a123-4482-b71c-2932f68a7bcb None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:10:51.133144 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-59e733a3-a123-4482-b71c-2932f68a7bcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:10:51.236433 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 663/1313] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:10:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:10:51.524630 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74565dd5-00ed-4fd8-a781-3f48d44102a0 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:10:51.526162 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-74565dd5-00ed-4fd8-a781-3f48d44102a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:10:51.632869 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74565dd5-00ed-4fd8-a781-3f48d44102a0 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:10:51.701177 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:10:51.703812 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee384cb-a118-4ddd-98c3-f974ec6657c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:10:51.704422 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee384cb-a118-4ddd-98c3-f974ec6657c9 service neutron] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:10:51.704701 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee384cb-a118-4ddd-98c3-f974ec6657c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:10:51.704972 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee384cb-a118-4ddd-98c3-f974ec6657c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:10:51.706609 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-0ee384cb-a118-4ddd-98c3-f974ec6657c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:10:51.813007 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee384cb-a118-4ddd-98c3-f974ec6657c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:10:51.814388 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee384cb-a118-4ddd-98c3-f974ec6657c9 service neutron] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:10:51.836690 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 651/1314] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:10:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:10:54.233028 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d44ce79-2dab-4d37-95b7-b7f1f818e405 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:10:54.233606 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d44ce79-2dab-4d37-95b7-b7f1f818e405 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:10:54.233606 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d44ce79-2dab-4d37-95b7-b7f1f818e405 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:10:54.234974 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 664/1315] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:10:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:10:54.240259 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de6e21-0971-4b7c-a6d8-71c2189d7a17 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:10:54.240663 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de6e21-0971-4b7c-a6d8-71c2189d7a17 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:10:54.241290 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de6e21-0971-4b7c-a6d8-71c2189d7a17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:10:54.304667 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-54de6e21-0971-4b7c-a6d8-71c2189d7a17 None None] MFA Rules not processed for user `26e7c9fc5d3c41de8156071d9f0cb3cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:10:54.305852 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-54de6e21-0971-4b7c-a6d8-71c2189d7a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:10:54.355275 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:10:54.398236 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 652/1316] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:10:54 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:10:54.678288 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d44ce79-2dab-4d37-95b7-b7f1f818e405 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:10:54.680433 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-8d44ce79-2dab-4d37-95b7-b7f1f818e405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:10:54.689984 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:10:54.783497 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d44ce79-2dab-4d37-95b7-b7f1f818e405 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:10:54.858144 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f11a9c-700e-417f-bdec-e96ea4012086 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:10:54.858785 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f11a9c-700e-417f-bdec-e96ea4012086 service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:10:54.859058 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f11a9c-700e-417f-bdec-e96ea4012086 service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:10:54.859326 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f11a9c-700e-417f-bdec-e96ea4012086 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:10:54.860913 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-24f11a9c-700e-417f-bdec-e96ea4012086 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:10:55.039752 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:10:55.087446 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f11a9c-700e-417f-bdec-e96ea4012086 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:10:55.088872 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f11a9c-700e-417f-bdec-e96ea4012086 service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:10:55.106582 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 665/1317] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:10:54 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:10:57.805197 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc14d187-78ee-4f37-b540-8e2ed48d0bfb None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:10:57.805713 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc14d187-78ee-4f37-b540-8e2ed48d0bfb None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:10:57.805713 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc14d187-78ee-4f37-b540-8e2ed48d0bfb None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:10:57.807041 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 653/1318] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:10:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:10:57.812428 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a79ee9d-7569-4a1e-8b1d-41de1871132e None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:10:57.812815 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a79ee9d-7569-4a1e-8b1d-41de1871132e None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:10:57.812966 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a79ee9d-7569-4a1e-8b1d-41de1871132e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:10:57.877535 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-3a79ee9d-7569-4a1e-8b1d-41de1871132e None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:10:57.878847 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-3a79ee9d-7569-4a1e-8b1d-41de1871132e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:10:57.974977 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 666/1319] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:10:57 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:10:58.267744 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc14d187-78ee-4f37-b540-8e2ed48d0bfb None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:10:58.269566 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-fc14d187-78ee-4f37-b540-8e2ed48d0bfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:10:58.373327 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc14d187-78ee-4f37-b540-8e2ed48d0bfb None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:10:58.444472 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c4f16d-78a9-425d-b6a3-12d547e44a13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:10:58.445188 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c4f16d-78a9-425d-b6a3-12d547e44a13 service neutron] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:10:58.445473 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c4f16d-78a9-425d-b6a3-12d547e44a13 service neutron] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:10:58.445747 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c4f16d-78a9-425d-b6a3-12d547e44a13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:10:58.447617 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-c7c4f16d-78a9-425d-b6a3-12d547e44a13 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:10:58.454990 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:10:58.554786 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c4f16d-78a9-425d-b6a3-12d547e44a13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:10:58.556153 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c4f16d-78a9-425d-b6a3-12d547e44a13 service neutron] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:10:58.575884 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 654/1320] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:10:58 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:11:01.412754 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990a1b42-3f61-4c0d-ab57-fe516e626527 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:11:01.413943 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990a1b42-3f61-4c0d-ab57-fe516e626527 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:11:01.413943 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990a1b42-3f61-4c0d-ab57-fe516e626527 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:11:01.415243 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 667/1321] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:11:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:11:01.420491 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af5d2c6-fe35-4e11-8922-59be13c4747f None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:11:01.420925 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af5d2c6-fe35-4e11-8922-59be13c4747f None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:11:01.421034 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af5d2c6-fe35-4e11-8922-59be13c4747f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:11:01.486821 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-6af5d2c6-fe35-4e11-8922-59be13c4747f None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:11:01.488033 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-6af5d2c6-fe35-4e11-8922-59be13c4747f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:11:01.546271 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:11:01.589495 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 655/1322] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:11:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:11:01.887863 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a2a199-9df1-4f9a-a31f-66ec8e587e8b None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:11:01.888261 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a2a199-9df1-4f9a-a31f-66ec8e587e8b None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:11:01.888607 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a2a199-9df1-4f9a-a31f-66ec8e587e8b None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:11:01.890063 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 668/1323] 10.222.0.58 () {54 vars in 965 bytes} [Thu Aug 25 03:11:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:11:01.896012 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96773b5b-dd6e-48f7-a7be-5d8f1c61b4a5 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:11:01.896296 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96773b5b-dd6e-48f7-a7be-5d8f1c61b4a5 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:11:01.896567 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96773b5b-dd6e-48f7-a7be-5d8f1c61b4a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:11:01.973688 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-96773b5b-dd6e-48f7-a7be-5d8f1c61b4a5 None None] MFA Rules not processed for user `a0a1a7f091774842907fd08de15e8058`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:11:01.974911 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-96773b5b-dd6e-48f7-a7be-5d8f1c61b4a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:11:02.048840 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 656/1324] 10.222.0.58 () {58 vars in 1062 bytes} [Thu Aug 25 03:11:01 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:11:02.057877 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a2a199-9df1-4f9a-a31f-66ec8e587e8b None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:11:02.059410 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-28a2a199-9df1-4f9a-a31f-66ec8e587e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:11:02.079624 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:11:02.164517 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a2a199-9df1-4f9a-a31f-66ec8e587e8b None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:11:02.239673 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a866f0d7-4269-4213-a567-f74e21b22793 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:11:02.240273 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a866f0d7-4269-4213-a567-f74e21b22793 service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:11:02.240552 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a866f0d7-4269-4213-a567-f74e21b22793 service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:11:02.240820 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a866f0d7-4269-4213-a567-f74e21b22793 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:11:02.242915 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-a866f0d7-4269-4213-a567-f74e21b22793 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:11:02.312463 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:11:02.349997 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a866f0d7-4269-4213-a567-f74e21b22793 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:11:02.351479 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a866f0d7-4269-4213-a567-f74e21b22793 service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:11:02.370038 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 669/1325] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:11:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 25 03:11:04.581249 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14457a17-c5f8-44d5-9d80-d5f12becca9b None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:11:04.581847 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14457a17-c5f8-44d5-9d80-d5f12becca9b None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:11:04.581847 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14457a17-c5f8-44d5-9d80-d5f12becca9b None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:11:04.583283 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 657/1326] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:11:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:11:04.589534 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56349e74-adbf-4ab0-877e-7354bb738861 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:11:04.590130 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56349e74-adbf-4ab0-877e-7354bb738861 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:11:04.590130 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56349e74-adbf-4ab0-877e-7354bb738861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:11:04.659343 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-56349e74-adbf-4ab0-877e-7354bb738861 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:11:04.660540 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-56349e74-adbf-4ab0-877e-7354bb738861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:11:04.766354 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 670/1327] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:11:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:11:05.051618 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14457a17-c5f8-44d5-9d80-d5f12becca9b None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:11:05.051618 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-14457a17-c5f8-44d5-9d80-d5f12becca9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:11:05.108830 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:11:05.152013 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14457a17-c5f8-44d5-9d80-d5f12becca9b None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:11:05.224025 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120e05c0-fa91-4485-911d-b63f7697755b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:11:05.224639 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120e05c0-fa91-4485-911d-b63f7697755b service neutron] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:11:05.224918 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120e05c0-fa91-4485-911d-b63f7697755b service neutron] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:11:05.225190 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120e05c0-fa91-4485-911d-b63f7697755b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:11:05.226851 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-120e05c0-fa91-4485-911d-b63f7697755b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:11:05.336469 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:11:05.338864 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120e05c0-fa91-4485-911d-b63f7697755b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:11:05.340308 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120e05c0-fa91-4485-911d-b63f7697755b service neutron] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:11:05.358299 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 658/1328] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:11:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:11:07.340265 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b052b3-f14a-4e4e-b457-db4757a922f5 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:11:07.340831 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b052b3-f14a-4e4e-b457-db4757a922f5 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:11:07.340831 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b052b3-f14a-4e4e-b457-db4757a922f5 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:11:07.346418 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 671/1329] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:11:07 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:11:07.348312 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9832e766-3f2f-45b0-bf66-1674783ad6b7 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:11:07.348742 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9832e766-3f2f-45b0-bf66-1674783ad6b7 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:11:07.348906 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9832e766-3f2f-45b0-bf66-1674783ad6b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:11:07.424262 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-9832e766-3f2f-45b0-bf66-1674783ad6b7 None None] MFA Rules not processed for user `26e7c9fc5d3c41de8156071d9f0cb3cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:11:07.425403 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-9832e766-3f2f-45b0-bf66-1674783ad6b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:11:07.532317 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 659/1330] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:11:07 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:11:07.824639 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b052b3-f14a-4e4e-b457-db4757a922f5 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:11:07.826194 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-12b052b3-f14a-4e4e-b457-db4757a922f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:11:07.857446 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:11:07.937938 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b052b3-f14a-4e4e-b457-db4757a922f5 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:11:08.025852 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371f04f0-9f68-4c51-8ec4-117c5b62ac83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:11:08.026575 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371f04f0-9f68-4c51-8ec4-117c5b62ac83 service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:11:08.026860 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371f04f0-9f68-4c51-8ec4-117c5b62ac83 service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:11:08.027135 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371f04f0-9f68-4c51-8ec4-117c5b62ac83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:11:08.028847 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-371f04f0-9f68-4c51-8ec4-117c5b62ac83 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:11:08.114085 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:11:08.146721 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371f04f0-9f68-4c51-8ec4-117c5b62ac83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:11:08.148865 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371f04f0-9f68-4c51-8ec4-117c5b62ac83 service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:11:08.168911 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 672/1331] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:11:07 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:11:10.550883 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3366be2d-70a0-4db9-b418-1f79e55c0c5a None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:11:10.551423 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3366be2d-70a0-4db9-b418-1f79e55c0c5a None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:11:10.551423 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3366be2d-70a0-4db9-b418-1f79e55c0c5a None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:11:10.552775 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 660/1332] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:11:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:11:10.557858 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a254ea9-160f-4242-b898-425f81320ade None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:11:10.558310 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a254ea9-160f-4242-b898-425f81320ade None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:11:10.558433 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a254ea9-160f-4242-b898-425f81320ade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:11:10.626633 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-1a254ea9-160f-4242-b898-425f81320ade None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:11:10.627841 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-1a254ea9-160f-4242-b898-425f81320ade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:11:10.722043 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 673/1333] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:11:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:11:11.003636 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3366be2d-70a0-4db9-b418-1f79e55c0c5a None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:11:11.005184 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-3366be2d-70a0-4db9-b418-1f79e55c0c5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:11:11.074542 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:11:11.107606 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3366be2d-70a0-4db9-b418-1f79e55c0c5a None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:11:11.179948 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a03c07-6360-46c6-82a2-d721c38a6967 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:11:11.180544 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a03c07-6360-46c6-82a2-d721c38a6967 service neutron] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:11:11.180820 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a03c07-6360-46c6-82a2-d721c38a6967 service neutron] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:11:11.181089 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a03c07-6360-46c6-82a2-d721c38a6967 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:11:11.182714 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-96a03c07-6360-46c6-82a2-d721c38a6967 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:11:11.286912 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a03c07-6360-46c6-82a2-d721c38a6967 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:11:11.288293 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a03c07-6360-46c6-82a2-d721c38a6967 service neutron] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:11:11.297808 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:11:11.306770 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 661/1334] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:11:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:11:13.269860 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae65c6b-96c6-41a8-81fa-af47750a383f None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:11:13.270446 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae65c6b-96c6-41a8-81fa-af47750a383f None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:11:13.270446 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae65c6b-96c6-41a8-81fa-af47750a383f None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:11:13.271793 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 674/1335] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:11:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:11:13.277563 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c999b8cb-01a5-46d4-a84c-273377449aa4 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:11:13.277998 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c999b8cb-01a5-46d4-a84c-273377449aa4 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:11:13.278128 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c999b8cb-01a5-46d4-a84c-273377449aa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:11:13.342906 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-c999b8cb-01a5-46d4-a84c-273377449aa4 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:11:13.344123 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-c999b8cb-01a5-46d4-a84c-273377449aa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:11:13.438807 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 662/1336] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:11:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:11:13.724276 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae65c6b-96c6-41a8-81fa-af47750a383f None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:11:13.725838 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-8ae65c6b-96c6-41a8-81fa-af47750a383f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:11:13.765780 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:11:13.828037 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae65c6b-96c6-41a8-81fa-af47750a383f None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:11:13.899159 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddea996-2284-4e84-9686-b6b825794579 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:11:13.899768 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddea996-2284-4e84-9686-b6b825794579 service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:11:13.900044 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddea996-2284-4e84-9686-b6b825794579 service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:11:13.900313 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddea996-2284-4e84-9686-b6b825794579 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:11:13.901873 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-cddea996-2284-4e84-9686-b6b825794579 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:11:13.986160 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:11:14.011610 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddea996-2284-4e84-9686-b6b825794579 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:11:14.013701 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddea996-2284-4e84-9686-b6b825794579 service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:11:14.032665 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 675/1337] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:11:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:11:16.565281 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2bbf2c-c613-43b1-88ca-2f73dc0d4960 None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:11:16.565820 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2bbf2c-c613-43b1-88ca-2f73dc0d4960 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:11:16.565820 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2bbf2c-c613-43b1-88ca-2f73dc0d4960 None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:11:16.567200 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 663/1338] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:11:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:11:16.572555 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db619c1b-a7e6-4a16-9c32-79e8738cf1c9 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:11:16.573008 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db619c1b-a7e6-4a16-9c32-79e8738cf1c9 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:11:16.573129 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db619c1b-a7e6-4a16-9c32-79e8738cf1c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:11:16.637660 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-db619c1b-a7e6-4a16-9c32-79e8738cf1c9 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:11:16.638879 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-db619c1b-a7e6-4a16-9c32-79e8738cf1c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:11:16.734479 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 676/1339] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:11:16 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:11:17.017360 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2bbf2c-c613-43b1-88ca-2f73dc0d4960 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:11:17.019005 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-ea2bbf2c-c613-43b1-88ca-2f73dc0d4960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:11:17.097094 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:11:17.126803 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2bbf2c-c613-43b1-88ca-2f73dc0d4960 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:11:17.199371 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14989ca-a612-4cad-a99b-1360d1cd2fe9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:11:17.199971 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14989ca-a612-4cad-a99b-1360d1cd2fe9 service neutron] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:11:17.200249 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14989ca-a612-4cad-a99b-1360d1cd2fe9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:11:17.200527 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14989ca-a612-4cad-a99b-1360d1cd2fe9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:11:17.202115 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-f14989ca-a612-4cad-a99b-1360d1cd2fe9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:11:17.305772 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14989ca-a612-4cad-a99b-1360d1cd2fe9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:11:17.307223 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14989ca-a612-4cad-a99b-1360d1cd2fe9 service neutron] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:11:17.325942 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 664/1340] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:11:17 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:11:21.463920 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dfd94d-f0e1-4b2c-ba38-83b4c36438bd None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:11:21.464429 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dfd94d-f0e1-4b2c-ba38-83b4c36438bd None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:11:21.464429 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dfd94d-f0e1-4b2c-ba38-83b4c36438bd None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:11:21.465741 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 677/1341] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:11:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:11:21.471062 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78b56b6-9c14-4538-8ad8-c75be6ad6577 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:11:21.471496 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78b56b6-9c14-4538-8ad8-c75be6ad6577 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:11:21.471606 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78b56b6-9c14-4538-8ad8-c75be6ad6577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:11:21.477579 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:11:21.538748 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-b78b56b6-9c14-4538-8ad8-c75be6ad6577 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:11:21.539982 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-b78b56b6-9c14-4538-8ad8-c75be6ad6577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:11:21.642995 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 665/1342] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:11:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:11:21.929324 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dfd94d-f0e1-4b2c-ba38-83b4c36438bd None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:11:21.930869 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-00dfd94d-f0e1-4b2c-ba38-83b4c36438bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:11:21.979781 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:11:22.036409 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dfd94d-f0e1-4b2c-ba38-83b4c36438bd None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:11:22.112939 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348d1e7d-0a8b-4b09-a3a9-588b447eef4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:11:22.113639 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348d1e7d-0a8b-4b09-a3a9-588b447eef4b service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:11:22.113925 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348d1e7d-0a8b-4b09-a3a9-588b447eef4b service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:11:22.114202 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348d1e7d-0a8b-4b09-a3a9-588b447eef4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:11:22.116437 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-348d1e7d-0a8b-4b09-a3a9-588b447eef4b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:11:22.210681 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:11:22.226858 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348d1e7d-0a8b-4b09-a3a9-588b447eef4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:11:22.228355 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348d1e7d-0a8b-4b09-a3a9-588b447eef4b service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:11:22.247947 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 678/1343] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:11:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:11:25.092105 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225e4c13-a1c8-45a4-9efe-0e0ce8c90f30 None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:11:25.092637 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225e4c13-a1c8-45a4-9efe-0e0ce8c90f30 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:11:25.092637 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225e4c13-a1c8-45a4-9efe-0e0ce8c90f30 None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:11:25.093988 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 666/1344] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:11:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:11:25.099284 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce2a2ca-a9eb-4739-8bdd-f639a30a6655 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:11:25.099733 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce2a2ca-a9eb-4739-8bdd-f639a30a6655 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:11:25.099842 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce2a2ca-a9eb-4739-8bdd-f639a30a6655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:11:25.166065 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-bce2a2ca-a9eb-4739-8bdd-f639a30a6655 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:11:25.167274 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-bce2a2ca-a9eb-4739-8bdd-f639a30a6655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:11:25.264429 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 679/1345] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:11:25 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:11:25.550976 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225e4c13-a1c8-45a4-9efe-0e0ce8c90f30 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:11:25.552500 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-225e4c13-a1c8-45a4-9efe-0e0ce8c90f30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:11:25.644155 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:11:25.661117 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225e4c13-a1c8-45a4-9efe-0e0ce8c90f30 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:11:25.736900 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d33c8ad-7510-4cba-a9b8-ef580d79e384 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:11:25.737537 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d33c8ad-7510-4cba-a9b8-ef580d79e384 service neutron] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:11:25.737818 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d33c8ad-7510-4cba-a9b8-ef580d79e384 service neutron] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:11:25.738088 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d33c8ad-7510-4cba-a9b8-ef580d79e384 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:11:25.739715 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-1d33c8ad-7510-4cba-a9b8-ef580d79e384 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:11:25.849547 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d33c8ad-7510-4cba-a9b8-ef580d79e384 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:11:25.851038 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d33c8ad-7510-4cba-a9b8-ef580d79e384 service neutron] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:11:25.869385 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 667/1346] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:11:25 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:11:30.528058 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2757ed12-f786-48ac-886e-5eb9adb46322 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:11:30.528543 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2757ed12-f786-48ac-886e-5eb9adb46322 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:11:30.528543 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2757ed12-f786-48ac-886e-5eb9adb46322 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:11:30.529855 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 680/1347] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:11:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 25 03:11:30.535546 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962d2ee5-f17a-4493-86a7-bff4347cb197 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:11:30.536146 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962d2ee5-f17a-4493-86a7-bff4347cb197 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:11:30.536146 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962d2ee5-f17a-4493-86a7-bff4347cb197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:11:30.552664 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:11:30.606976 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-962d2ee5-f17a-4493-86a7-bff4347cb197 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:11:30.608220 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-962d2ee5-f17a-4493-86a7-bff4347cb197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:11:30.713635 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 668/1348] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:11:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:11:30.997724 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2757ed12-f786-48ac-886e-5eb9adb46322 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:11:30.999366 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-2757ed12-f786-48ac-886e-5eb9adb46322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:11:31.056724 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:11:31.102365 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2757ed12-f786-48ac-886e-5eb9adb46322 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:11:31.174178 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dab759c-18e1-47c8-b28f-93a0e8ea310a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:11:31.174809 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dab759c-18e1-47c8-b28f-93a0e8ea310a service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:11:31.175091 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dab759c-18e1-47c8-b28f-93a0e8ea310a service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:11:31.175367 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dab759c-18e1-47c8-b28f-93a0e8ea310a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:11:31.176991 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-5dab759c-18e1-47c8-b28f-93a0e8ea310a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:11:31.280631 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:11:31.283979 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dab759c-18e1-47c8-b28f-93a0e8ea310a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:11:31.285400 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dab759c-18e1-47c8-b28f-93a0e8ea310a service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:11:31.303555 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 681/1349] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:11:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:11:36.263484 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc45b8d-eadf-411e-8d88-a99c2e960956 None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:11:36.264500 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc45b8d-eadf-411e-8d88-a99c2e960956 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:11:36.264946 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc45b8d-eadf-411e-8d88-a99c2e960956 None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:11:36.266768 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 669/1350] 10.222.0.58 () {54 vars in 921 bytes} [Thu Aug 25 03:11:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:11:36.274240 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567a59f5-75b2-447d-bcd9-b5e7aa7fee38 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:11:36.274793 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567a59f5-75b2-447d-bcd9-b5e7aa7fee38 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:11:36.275253 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567a59f5-75b2-447d-bcd9-b5e7aa7fee38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:11:36.407299 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-567a59f5-75b2-447d-bcd9-b5e7aa7fee38 None None] MFA Rules not processed for user `2b692edd3e8c41289cc42e1afb4a6dd8`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:11:36.407299 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-567a59f5-75b2-447d-bcd9-b5e7aa7fee38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:11:36.533856 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 682/1351] 10.222.0.58 () {58 vars in 1018 bytes} [Thu Aug 25 03:11:36 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:11:40.714882 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=59 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:11:40.807594 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693a1833-dcf5-4337-813d-2b6b00af8280 None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:11:40.807884 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693a1833-dcf5-4337-813d-2b6b00af8280 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:11:40.808332 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693a1833-dcf5-4337-813d-2b6b00af8280 None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:11:40.810089 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 670/1352] 10.222.0.58 () {54 vars in 921 bytes} [Thu Aug 25 03:11:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:11:40.815296 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa4f088-99d4-425a-ab0a-ae02c2b47fdb None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:11:40.816104 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa4f088-99d4-425a-ab0a-ae02c2b47fdb None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:11:40.816555 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa4f088-99d4-425a-ab0a-ae02c2b47fdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:11:40.875319 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:11:40.907130 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-1fa4f088-99d4-425a-ab0a-ae02c2b47fdb None None] MFA Rules not processed for user `2b692edd3e8c41289cc42e1afb4a6dd8`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:11:40.908848 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-1fa4f088-99d4-425a-ab0a-ae02c2b47fdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:11:41.038879 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 683/1353] 10.222.0.58 () {58 vars in 1018 bytes} [Thu Aug 25 03:11:40 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:11:42.347563 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fef08bb-3186-43aa-b273-5b636cfee5ec None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:11:42.348123 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fef08bb-3186-43aa-b273-5b636cfee5ec None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:11:42.348237 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fef08bb-3186-43aa-b273-5b636cfee5ec None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:11:42.349637 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 671/1354] 10.222.0.58 () {54 vars in 921 bytes} [Thu Aug 25 03:11:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:11:42.354783 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9358645a-491e-44aa-ba2d-7dcbcdb04f7d None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:11:42.355184 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9358645a-491e-44aa-ba2d-7dcbcdb04f7d None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:11:42.355325 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9358645a-491e-44aa-ba2d-7dcbcdb04f7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:11:42.437777 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-9358645a-491e-44aa-ba2d-7dcbcdb04f7d None None] MFA Rules not processed for user `2b692edd3e8c41289cc42e1afb4a6dd8`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:11:42.439864 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-9358645a-491e-44aa-ba2d-7dcbcdb04f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:11:42.496189 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:11:42.562250 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 684/1355] 10.222.0.58 () {58 vars in 1018 bytes} [Thu Aug 25 03:11:42 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:11:46.106091 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959f169b-c0d0-413d-a8d0-9cea6c254e82 None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:11:46.106786 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959f169b-c0d0-413d-a8d0-9cea6c254e82 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:11:46.106786 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959f169b-c0d0-413d-a8d0-9cea6c254e82 None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:11:46.108251 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 672/1356] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:11:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:11:46.113754 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771f24a0-b1ff-4caf-8d72-872350e600f4 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:11:46.114244 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771f24a0-b1ff-4caf-8d72-872350e600f4 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:11:46.114360 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771f24a0-b1ff-4caf-8d72-872350e600f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:11:46.195808 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-771f24a0-b1ff-4caf-8d72-872350e600f4 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:11:46.197296 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-771f24a0-b1ff-4caf-8d72-872350e600f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:11:46.328748 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 685/1357] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:11:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:11:46.637216 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f72afc-e638-40c5-9eba-5cca5e340c32 None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:11:46.637615 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f72afc-e638-40c5-9eba-5cca5e340c32 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:11:46.638113 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f72afc-e638-40c5-9eba-5cca5e340c32 None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:11:46.639621 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 673/1358] 10.222.0.58 () {54 vars in 962 bytes} [Thu Aug 25 03:11:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:11:46.647285 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdc27e9-a636-46bd-8fab-3b0cdbe85e8d None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:11:46.647672 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdc27e9-a636-46bd-8fab-3b0cdbe85e8d None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:11:46.648019 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdc27e9-a636-46bd-8fab-3b0cdbe85e8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:11:46.665121 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:11:46.737141 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-7cdc27e9-a636-46bd-8fab-3b0cdbe85e8d None None] MFA Rules not processed for user `0cd171bf4df94f8fab397e42835feeab`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:11:46.738519 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-7cdc27e9-a636-46bd-8fab-3b0cdbe85e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:11:46.863628 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 686/1359] 10.222.0.58 () {58 vars in 1059 bytes} [Thu Aug 25 03:11:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:11:46.871711 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f72afc-e638-40c5-9eba-5cca5e340c32 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:11:46.873740 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-d1f72afc-e638-40c5-9eba-5cca5e340c32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:11:47.029971 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f72afc-e638-40c5-9eba-5cca5e340c32 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:11:47.108876 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:11:47.152311 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3cb756-e4e1-4785-9d06-d71a7149668d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cd171bf4df94f8fab397e42835feeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:11:47.153218 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3cb756-e4e1-4785-9d06-d71a7149668d service nova] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:11:47.153500 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3cb756-e4e1-4785-9d06-d71a7149668d service nova] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:11:47.153772 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3cb756-e4e1-4785-9d06-d71a7149668d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:11:47.155444 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-5d3cb756-e4e1-4785-9d06-d71a7149668d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:11:47.248253 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3cb756-e4e1-4785-9d06-d71a7149668d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:11:47.249907 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3cb756-e4e1-4785-9d06-d71a7149668d service nova] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:11:47.268782 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 674/1360] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:11:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:11:49.404646 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b68ac9c-59cc-438e-8586-c650cde9ab52 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:11:49.405095 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b68ac9c-59cc-438e-8586-c650cde9ab52 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:11:49.405198 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b68ac9c-59cc-438e-8586-c650cde9ab52 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:11:49.406762 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 687/1361] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:11:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:11:49.411816 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889bceac-44e3-42a3-88d4-ad69d60b49f9 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:11:49.412233 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889bceac-44e3-42a3-88d4-ad69d60b49f9 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:11:49.413066 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889bceac-44e3-42a3-88d4-ad69d60b49f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:11:49.483042 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-889bceac-44e3-42a3-88d4-ad69d60b49f9 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:11:49.484285 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-889bceac-44e3-42a3-88d4-ad69d60b49f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:11:49.524782 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:11:49.589931 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 675/1362] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:11:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:11:49.902942 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b68ac9c-59cc-438e-8586-c650cde9ab52 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:11:49.904633 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-5b68ac9c-59cc-438e-8586-c650cde9ab52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:11:49.988249 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:11:50.086208 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b68ac9c-59cc-438e-8586-c650cde9ab52 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:11:50.204208 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b59a22-a09c-4514-840b-c145f1cd4f2c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cd171bf4df94f8fab397e42835feeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:11:50.204817 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b59a22-a09c-4514-840b-c145f1cd4f2c service nova] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:11:50.205097 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b59a22-a09c-4514-840b-c145f1cd4f2c service nova] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:11:50.205369 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b59a22-a09c-4514-840b-c145f1cd4f2c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:11:50.206992 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-31b59a22-a09c-4514-840b-c145f1cd4f2c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:11:50.230273 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:11:50.298565 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b59a22-a09c-4514-840b-c145f1cd4f2c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:11:50.300059 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b59a22-a09c-4514-840b-c145f1cd4f2c service nova] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:11:50.329718 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 688/1363] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:11:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:11:52.733286 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986e942f-3566-4410-ac09-27ff3398128d None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:11:52.733809 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986e942f-3566-4410-ac09-27ff3398128d None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:11:52.733809 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986e942f-3566-4410-ac09-27ff3398128d None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:11:52.735379 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 676/1364] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:11:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:11:52.741518 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42581db2-d294-4ddd-bea0-3f49f27264cb None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:11:52.741954 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42581db2-d294-4ddd-bea0-3f49f27264cb None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:11:52.742141 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42581db2-d294-4ddd-bea0-3f49f27264cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:11:52.813709 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-42581db2-d294-4ddd-bea0-3f49f27264cb None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:11:52.818995 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-42581db2-d294-4ddd-bea0-3f49f27264cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:11:52.917035 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:11:52.926311 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 689/1365] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:11:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:11:53.218174 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1818ffab-f58c-4a5c-813b-1716c421ed54 None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:11:53.218674 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1818ffab-f58c-4a5c-813b-1716c421ed54 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:11:53.219030 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1818ffab-f58c-4a5c-813b-1716c421ed54 None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:11:53.220807 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 677/1366] 10.222.0.58 () {54 vars in 962 bytes} [Thu Aug 25 03:11:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:11:53.226989 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bce4b6-9ab2-476d-b539-95e473643b0f None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:11:53.227470 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bce4b6-9ab2-476d-b539-95e473643b0f None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:11:53.227824 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bce4b6-9ab2-476d-b539-95e473643b0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:11:53.308781 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-24bce4b6-9ab2-476d-b539-95e473643b0f None None] MFA Rules not processed for user `0cd171bf4df94f8fab397e42835feeab`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:11:53.310030 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-24bce4b6-9ab2-476d-b539-95e473643b0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:11:53.435128 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 690/1367] 10.222.0.58 () {58 vars in 1059 bytes} [Thu Aug 25 03:11:53 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:11:53.442811 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1818ffab-f58c-4a5c-813b-1716c421ed54 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:11:53.444377 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-1818ffab-f58c-4a5c-813b-1716c421ed54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:11:53.606143 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1818ffab-f58c-4a5c-813b-1716c421ed54 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:11:53.627245 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:11:53.727351 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9370f5d0-03bc-462d-88ad-ac96d9c81651 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cd171bf4df94f8fab397e42835feeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:11:53.727964 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9370f5d0-03bc-462d-88ad-ac96d9c81651 service nova] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:11:53.728242 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9370f5d0-03bc-462d-88ad-ac96d9c81651 service nova] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:11:53.728518 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9370f5d0-03bc-462d-88ad-ac96d9c81651 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:11:53.730174 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-9370f5d0-03bc-462d-88ad-ac96d9c81651 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:11:53.827138 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9370f5d0-03bc-462d-88ad-ac96d9c81651 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:11:53.828757 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9370f5d0-03bc-462d-88ad-ac96d9c81651 service nova] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:11:53.847828 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 678/1368] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:11:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:11:54.392305 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812df3b7-dd2d-44b8-bc59-d8ccd50e12d3 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:11:54.392843 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812df3b7-dd2d-44b8-bc59-d8ccd50e12d3 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:11:54.392843 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812df3b7-dd2d-44b8-bc59-d8ccd50e12d3 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:11:54.394426 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 691/1369] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:11:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:11:54.399950 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befaae57-c508-4875-8a16-35eaaae45e31 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:11:54.400262 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befaae57-c508-4875-8a16-35eaaae45e31 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:11:54.400535 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befaae57-c508-4875-8a16-35eaaae45e31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:11:54.436437 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:11:54.474404 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-befaae57-c508-4875-8a16-35eaaae45e31 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:11:54.476140 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-befaae57-c508-4875-8a16-35eaaae45e31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:11:54.588924 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 679/1370] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:11:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:11:55.195863 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d85106-3b51-45e3-b33f-f144b1be0c76 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:11:55.195863 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d85106-3b51-45e3-b33f-f144b1be0c76 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:11:55.195863 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d85106-3b51-45e3-b33f-f144b1be0c76 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:11:55.197134 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 692/1371] 10.222.0.58 () {54 vars in 964 bytes} [Thu Aug 25 03:11:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:11:55.202437 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5dc53c-b0a9-4a5a-8e2c-347802dd69af None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:11:55.202944 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5dc53c-b0a9-4a5a-8e2c-347802dd69af None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:11:55.203116 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5dc53c-b0a9-4a5a-8e2c-347802dd69af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:11:55.279649 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-4b5dc53c-b0a9-4a5a-8e2c-347802dd69af None None] MFA Rules not processed for user `60193ba9e1c14df3bc9eebcc6f28f2d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:11:55.280869 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-4b5dc53c-b0a9-4a5a-8e2c-347802dd69af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:11:55.329444 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:11:55.360037 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 680/1372] 10.222.0.58 () {58 vars in 1061 bytes} [Thu Aug 25 03:11:55 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:11:55.367346 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d85106-3b51-45e3-b33f-f144b1be0c76 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:11:55.368842 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-f5d85106-3b51-45e3-b33f-f144b1be0c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:11:55.420205 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:11:55.475941 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d85106-3b51-45e3-b33f-f144b1be0c76 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:11:55.553529 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e7c65e-909e-4bcd-871d-101f89a9da71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60193ba9e1c14df3bc9eebcc6f28f2d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:11:55.554142 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e7c65e-909e-4bcd-871d-101f89a9da71 service cinder] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:11:55.554452 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e7c65e-909e-4bcd-871d-101f89a9da71 service cinder] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:11:55.554723 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e7c65e-909e-4bcd-871d-101f89a9da71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:11:55.556330 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-a0e7c65e-909e-4bcd-871d-101f89a9da71 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:11:55.674267 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:11:55.689151 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e7c65e-909e-4bcd-871d-101f89a9da71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:11:55.690637 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e7c65e-909e-4bcd-871d-101f89a9da71 service cinder] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:11:55.713078 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 693/1373] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:11:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:11:56.043176 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73af5de-9887-4941-b7ac-b77cfb160e11 None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:11:56.043459 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73af5de-9887-4941-b7ac-b77cfb160e11 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:11:56.044103 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73af5de-9887-4941-b7ac-b77cfb160e11 None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:11:56.046035 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 681/1374] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:11:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:11:56.050948 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35694bb9-32a0-4573-99c8-bfd21efdd9b0 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:11:56.051234 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35694bb9-32a0-4573-99c8-bfd21efdd9b0 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:11:56.056710 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35694bb9-32a0-4573-99c8-bfd21efdd9b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:11:56.120014 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-35694bb9-32a0-4573-99c8-bfd21efdd9b0 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:11:56.121227 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-35694bb9-32a0-4573-99c8-bfd21efdd9b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:11:56.216508 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 694/1375] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:11:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:11:56.508381 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73af5de-9887-4941-b7ac-b77cfb160e11 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:11:56.509879 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-e73af5de-9887-4941-b7ac-b77cfb160e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:11:56.670449 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73af5de-9887-4941-b7ac-b77cfb160e11 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:11:56.724168 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:11:56.788832 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def2b126-e199-4879-8756-d2fef13d9332 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cd171bf4df94f8fab397e42835feeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:11:56.789448 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def2b126-e199-4879-8756-d2fef13d9332 service nova] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:11:56.789749 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def2b126-e199-4879-8756-d2fef13d9332 service nova] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:11:56.789999 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def2b126-e199-4879-8756-d2fef13d9332 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:11:56.791659 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-def2b126-e199-4879-8756-d2fef13d9332 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:11:56.880042 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def2b126-e199-4879-8756-d2fef13d9332 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:11:56.882141 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def2b126-e199-4879-8756-d2fef13d9332 service nova] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:11:56.900927 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 682/1376] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:11:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:11:58.969768 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f66a9f-699b-48bc-aabb-46c6003034a7 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:11:58.970321 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f66a9f-699b-48bc-aabb-46c6003034a7 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:11:58.970321 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f66a9f-699b-48bc-aabb-46c6003034a7 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:11:58.971601 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 695/1377] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:11:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:11:58.977537 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb301f09-4859-4d3e-b642-3bcd7486a1f1 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:11:58.977993 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb301f09-4859-4d3e-b642-3bcd7486a1f1 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:11:58.978236 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb301f09-4859-4d3e-b642-3bcd7486a1f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:11:59.043375 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-fb301f09-4859-4d3e-b642-3bcd7486a1f1 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:11:59.044576 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-fb301f09-4859-4d3e-b642-3bcd7486a1f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:11:59.051159 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:11:59.141504 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 683/1378] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:11:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:11:59.437798 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f66a9f-699b-48bc-aabb-46c6003034a7 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:11:59.440223 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-c6f66a9f-699b-48bc-aabb-46c6003034a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:11:59.520926 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:11:59.621802 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f66a9f-699b-48bc-aabb-46c6003034a7 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:11:59.762328 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02cbd36-7b77-42e9-abbb-7d6a463b0a21 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cd171bf4df94f8fab397e42835feeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:11:59.763073 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02cbd36-7b77-42e9-abbb-7d6a463b0a21 service nova] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:11:59.763363 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02cbd36-7b77-42e9-abbb-7d6a463b0a21 service nova] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:11:59.763627 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02cbd36-7b77-42e9-abbb-7d6a463b0a21 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:11:59.765275 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-c02cbd36-7b77-42e9-abbb-7d6a463b0a21 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:11:59.786726 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:11:59.898033 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02cbd36-7b77-42e9-abbb-7d6a463b0a21 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:11:59.898033 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02cbd36-7b77-42e9-abbb-7d6a463b0a21 service nova] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:11:59.898033 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 696/1379] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:11:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:12:02.054677 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655b58d5-337f-4d22-9175-70470aa300fb None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:02.054677 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655b58d5-337f-4d22-9175-70470aa300fb None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:02.054677 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655b58d5-337f-4d22-9175-70470aa300fb None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:02.056182 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 684/1380] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:12:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:12:02.059572 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461bd381-d304-4e79-a30d-13c24ad64fab None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:02.059994 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461bd381-d304-4e79-a30d-13c24ad64fab None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:02.060122 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461bd381-d304-4e79-a30d-13c24ad64fab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:02.130437 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-461bd381-d304-4e79-a30d-13c24ad64fab None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:12:02.131686 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-461bd381-d304-4e79-a30d-13c24ad64fab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:12:02.240649 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:12:02.247876 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 697/1381] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:12:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:12:02.542442 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655b58d5-337f-4d22-9175-70470aa300fb None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:12:02.544330 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-655b58d5-337f-4d22-9175-70470aa300fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:12:02.700493 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:12:02.712277 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655b58d5-337f-4d22-9175-70470aa300fb None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:12:02.831761 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b60f09e-415d-4803-bad1-2f9eafd3e19f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cd171bf4df94f8fab397e42835feeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:12:02.832374 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b60f09e-415d-4803-bad1-2f9eafd3e19f service nova] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:02.832709 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b60f09e-415d-4803-bad1-2f9eafd3e19f service nova] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:02.832988 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b60f09e-415d-4803-bad1-2f9eafd3e19f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:02.835158 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-9b60f09e-415d-4803-bad1-2f9eafd3e19f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:12:02.928447 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b60f09e-415d-4803-bad1-2f9eafd3e19f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:12:02.930062 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b60f09e-415d-4803-bad1-2f9eafd3e19f service nova] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:12:02.942746 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:12:02.952216 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 685/1382] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:12:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:12:05.345951 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01cc770-e53d-4d80-9693-de2ff3ac5ec8 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:05.346544 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01cc770-e53d-4d80-9693-de2ff3ac5ec8 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:05.346858 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01cc770-e53d-4d80-9693-de2ff3ac5ec8 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:05.348611 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 698/1383] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:12:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:12:05.354818 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fd997f-4f4a-40ab-907f-32e4d226ef1a None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:05.355213 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fd997f-4f4a-40ab-907f-32e4d226ef1a None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:05.355394 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fd997f-4f4a-40ab-907f-32e4d226ef1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:05.427961 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-25fd997f-4f4a-40ab-907f-32e4d226ef1a None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:12:05.429169 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-25fd997f-4f4a-40ab-907f-32e4d226ef1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:12:05.536431 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 686/1384] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:12:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:12:05.841412 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01cc770-e53d-4d80-9693-de2ff3ac5ec8 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:12:05.843432 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-b01cc770-e53d-4d80-9693-de2ff3ac5ec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:12:06.011780 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01cc770-e53d-4d80-9693-de2ff3ac5ec8 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:12:06.086245 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:12:06.133443 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecb3356-7e2b-4f74-bf6f-93ddb8fd3c77 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cd171bf4df94f8fab397e42835feeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:12:06.134767 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecb3356-7e2b-4f74-bf6f-93ddb8fd3c77 service nova] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:06.135060 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecb3356-7e2b-4f74-bf6f-93ddb8fd3c77 service nova] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:06.135342 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecb3356-7e2b-4f74-bf6f-93ddb8fd3c77 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:06.137054 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-8ecb3356-7e2b-4f74-bf6f-93ddb8fd3c77 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:12:06.231951 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecb3356-7e2b-4f74-bf6f-93ddb8fd3c77 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:12:06.233681 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecb3356-7e2b-4f74-bf6f-93ddb8fd3c77 service nova] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:12:06.252839 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 699/1385] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:12:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:12:08.166121 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f65fbc-df3e-4dfa-aa7d-b49421b486e5 None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:08.166696 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f65fbc-df3e-4dfa-aa7d-b49421b486e5 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:08.166696 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f65fbc-df3e-4dfa-aa7d-b49421b486e5 None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:08.168199 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 687/1386] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:12:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 25 03:12:08.175502 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7759ad4f-a18f-4720-ac7c-b7467ba14dfa None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:08.175950 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7759ad4f-a18f-4720-ac7c-b7467ba14dfa None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:08.176240 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7759ad4f-a18f-4720-ac7c-b7467ba14dfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:08.247632 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-7759ad4f-a18f-4720-ac7c-b7467ba14dfa None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:12:08.248836 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-7759ad4f-a18f-4720-ac7c-b7467ba14dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:12:08.281966 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:12:08.350163 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 700/1387] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:12:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:12:08.644638 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f65fbc-df3e-4dfa-aa7d-b49421b486e5 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:12:08.646244 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-d2f65fbc-df3e-4dfa-aa7d-b49421b486e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:12:08.725641 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:12:08.796072 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f65fbc-df3e-4dfa-aa7d-b49421b486e5 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:12:08.912472 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfb8a1d-3afe-4a7b-af21-e9304a012f47 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cd171bf4df94f8fab397e42835feeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:12:08.913086 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfb8a1d-3afe-4a7b-af21-e9304a012f47 service nova] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:08.913987 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfb8a1d-3afe-4a7b-af21-e9304a012f47 service nova] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:08.914261 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfb8a1d-3afe-4a7b-af21-e9304a012f47 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:08.915899 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-4cfb8a1d-3afe-4a7b-af21-e9304a012f47 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:12:08.953529 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:12:09.003807 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfb8a1d-3afe-4a7b-af21-e9304a012f47 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:12:09.005488 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfb8a1d-3afe-4a7b-af21-e9304a012f47 service nova] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:12:09.023860 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 688/1388] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:12:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:12:11.019997 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d805ab8-71d9-4985-ab65-2c6fe2c3377c None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:11.020618 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d805ab8-71d9-4985-ab65-2c6fe2c3377c None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:11.020618 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d805ab8-71d9-4985-ab65-2c6fe2c3377c None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:11.021888 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 701/1389] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:12:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:12:11.027708 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca34531-9809-4761-b433-a853b54da838 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:11.028124 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca34531-9809-4761-b433-a853b54da838 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:11.028271 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca34531-9809-4761-b433-a853b54da838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:11.091843 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6657ec9-5b24-4244-85c0-7e4f12829c49 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:11.092124 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6657ec9-5b24-4244-85c0-7e4f12829c49 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:11.092403 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6657ec9-5b24-4244-85c0-7e4f12829c49 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:11.093739 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 702/1390] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:12:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 25 03:12:11.095976 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-6ca34531-9809-4761-b433-a853b54da838 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:12:11.097314 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-6ca34531-9809-4761-b433-a853b54da838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:12:11.098896 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5f30cc-a027-4c98-b21a-598b9f756e7a None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:11.099255 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5f30cc-a027-4c98-b21a-598b9f756e7a None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:11.099540 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5f30cc-a027-4c98-b21a-598b9f756e7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:11.165926 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-0d5f30cc-a027-4c98-b21a-598b9f756e7a None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:12:11.167203 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-0d5f30cc-a027-4c98-b21a-598b9f756e7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:12:11.200271 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 689/1391] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:12:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:12:11.266913 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 703/1392] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:12:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:12:11.500316 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ac8c58-2e98-43b5-bac3-7d2d2c9e898c None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:11.500605 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ac8c58-2e98-43b5-bac3-7d2d2c9e898c None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:11.500882 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ac8c58-2e98-43b5-bac3-7d2d2c9e898c None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:11.502214 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 690/1393] 10.222.0.58 () {54 vars in 967 bytes} [Thu Aug 25 03:12:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:12:11.507641 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63a8a21-f004-4a89-813f-60f53697e3e4 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:11.507929 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63a8a21-f004-4a89-813f-60f53697e3e4 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:11.508204 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63a8a21-f004-4a89-813f-60f53697e3e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:11.524742 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:12:11.563645 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ac8c58-2e98-43b5-bac3-7d2d2c9e898c None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:12:11.565158 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-d4ac8c58-2e98-43b5-bac3-7d2d2c9e898c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:12:11.580694 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:12:11.586779 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-c63a8a21-f004-4a89-813f-60f53697e3e4 None None] MFA Rules not processed for user `3f89089b390c4b6599e54000c0fa082a`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:12:11.593859 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-c63a8a21-f004-4a89-813f-60f53697e3e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:12:11.674004 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 704/1394] 10.222.0.58 () {58 vars in 1064 bytes} [Thu Aug 25 03:12:11 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:12:11.681880 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63a8a21-f004-4a89-813f-60f53697e3e4 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:12:11.689262 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63a8a21-f004-4a89-813f-60f53697e3e4 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:12:11.718862 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ac8c58-2e98-43b5-bac3-7d2d2c9e898c None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:12:11.765393 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec762d0-15c4-4a06-992c-20bd670809f0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f89089b390c4b6599e54000c0fa082a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:12:11.765928 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec762d0-15c4-4a06-992c-20bd670809f0 service glance] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:11.766239 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec762d0-15c4-4a06-992c-20bd670809f0 service glance] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:11.766550 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec762d0-15c4-4a06-992c-20bd670809f0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:11.768472 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-6ec762d0-15c4-4a06-992c-20bd670809f0 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:12:11.802748 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:12:11.890074 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec762d0-15c4-4a06-992c-20bd670809f0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:12:11.891545 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec762d0-15c4-4a06-992c-20bd670809f0 service glance] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:12:11.910065 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 705/1395] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:12:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 25 03:12:11.921209 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:12:11.949535 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb379713-f236-4af1-b12f-82815dfe168a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cd171bf4df94f8fab397e42835feeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:12:11.950147 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb379713-f236-4af1-b12f-82815dfe168a service nova] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:11.950462 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb379713-f236-4af1-b12f-82815dfe168a service nova] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:11.950747 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb379713-f236-4af1-b12f-82815dfe168a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:11.952355 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-eb379713-f236-4af1-b12f-82815dfe168a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:12:12.040989 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb379713-f236-4af1-b12f-82815dfe168a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:12:12.042555 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb379713-f236-4af1-b12f-82815dfe168a service nova] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:12:12.060355 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 691/1396] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:12:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 25 03:12:12.758882 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6d1271-4192-4c16-8ca7-9b2e6f5558e8 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:12.759365 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6d1271-4192-4c16-8ca7-9b2e6f5558e8 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:12.759469 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6d1271-4192-4c16-8ca7-9b2e6f5558e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:12.826273 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-3d6d1271-4192-4c16-8ca7-9b2e6f5558e8 None None] MFA Rules not processed for user `712c35b765b64919aa3eaaf02892590d`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:12:12.827503 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-3d6d1271-4192-4c16-8ca7-9b2e6f5558e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:12:12.906954 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:12:12.913901 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 706/1397] 10.222.0.58 () {58 vars in 1009 bytes} [Thu Aug 25 03:12:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:12:12.956728 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a765f89-d8ce-4dc4-95dd-cdd2a38a524b None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:12.957088 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a765f89-d8ce-4dc4-95dd-cdd2a38a524b None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:12.957434 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a765f89-d8ce-4dc4-95dd-cdd2a38a524b None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:12.958913 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 692/1398] 10.222.0.58 () {54 vars in 943 bytes} [Thu Aug 25 03:12:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:12:12.965191 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac8fc5e-e2d9-4fe6-8428-5dd29736ff57 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:12.965506 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac8fc5e-e2d9-4fe6-8428-5dd29736ff57 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:12.965784 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac8fc5e-e2d9-4fe6-8428-5dd29736ff57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:13.041989 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-fac8fc5e-e2d9-4fe6-8428-5dd29736ff57 None None] MFA Rules not processed for user `5047e1a7cd2d416d90b2d8c5a6596522`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:12:13.043220 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-fac8fc5e-e2d9-4fe6-8428-5dd29736ff57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:12:13.150056 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 707/1399] 10.222.0.58 () {58 vars in 1040 bytes} [Thu Aug 25 03:12:12 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:12:13.159450 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a765f89-d8ce-4dc4-95dd-cdd2a38a524b None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:12:13.160904 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-9a765f89-d8ce-4dc4-95dd-cdd2a38a524b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:12:13.255631 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:12:13.295955 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a765f89-d8ce-4dc4-95dd-cdd2a38a524b None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:12:13.402098 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0e0300-e9ce-48c7-8680-fca1d3916914 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5047e1a7cd2d416d90b2d8c5a6596522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:12:13.402740 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0e0300-e9ce-48c7-8680-fca1d3916914 service swift] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:13.403281 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0e0300-e9ce-48c7-8680-fca1d3916914 service swift] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:13.403558 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0e0300-e9ce-48c7-8680-fca1d3916914 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:13.405174 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-8b0e0300-e9ce-48c7-8680-fca1d3916914 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:12:13.491578 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0e0300-e9ce-48c7-8680-fca1d3916914 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:12:13.492986 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0e0300-e9ce-48c7-8680-fca1d3916914 service swift] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:12:13.496453 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:12:13.501606 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 693/1400] 10.222.0.58 () {60 vars in 1384 bytes} [Thu Aug 25 03:12:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 25 03:12:14.194273 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32354191-07a9-4c96-a46c-07ae7c3d50fa None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:14.195401 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32354191-07a9-4c96-a46c-07ae7c3d50fa None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:14.195485 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32354191-07a9-4c96-a46c-07ae7c3d50fa None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:14.197068 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 708/1401] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:12:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:12:14.202575 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20ee9f9-9407-4214-a7cc-94cbd6d61e30 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:14.203016 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20ee9f9-9407-4214-a7cc-94cbd6d61e30 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:14.203193 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20ee9f9-9407-4214-a7cc-94cbd6d61e30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:14.275861 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-f20ee9f9-9407-4214-a7cc-94cbd6d61e30 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:12:14.277285 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-f20ee9f9-9407-4214-a7cc-94cbd6d61e30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:12:14.386887 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 694/1402] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:12:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:12:14.680850 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32354191-07a9-4c96-a46c-07ae7c3d50fa None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:12:14.682601 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-32354191-07a9-4c96-a46c-07ae7c3d50fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:12:14.753341 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:12:14.872909 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32354191-07a9-4c96-a46c-07ae7c3d50fa None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:12:15.009873 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa21b7b-a7d3-4d34-a513-90099ad952de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cd171bf4df94f8fab397e42835feeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:12:15.010442 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa21b7b-a7d3-4d34-a513-90099ad952de service nova] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:15.010728 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa21b7b-a7d3-4d34-a513-90099ad952de service nova] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:15.010996 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa21b7b-a7d3-4d34-a513-90099ad952de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:15.012607 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-7aa21b7b-a7d3-4d34-a513-90099ad952de service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:12:15.031774 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:12:15.113308 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa21b7b-a7d3-4d34-a513-90099ad952de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:12:15.115034 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa21b7b-a7d3-4d34-a513-90099ad952de service nova] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:12:15.136421 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 709/1403] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:12:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:12:17.445363 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd6fc6f-d4a5-462f-8788-27fe8de8f0f5 None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:17.445962 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd6fc6f-d4a5-462f-8788-27fe8de8f0f5 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:17.446472 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd6fc6f-d4a5-462f-8788-27fe8de8f0f5 None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:17.448008 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 695/1404] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:12:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:12:17.453691 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65567626-d1f2-4325-9c81-2ed3af01b287 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:17.454219 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65567626-d1f2-4325-9c81-2ed3af01b287 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:17.454219 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65567626-d1f2-4325-9c81-2ed3af01b287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:17.530173 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-65567626-d1f2-4325-9c81-2ed3af01b287 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:12:17.531416 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-65567626-d1f2-4325-9c81-2ed3af01b287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:12:17.641819 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:12:17.658324 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 710/1405] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:12:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:12:17.945585 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd6fc6f-d4a5-462f-8788-27fe8de8f0f5 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:12:17.947966 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-0dd6fc6f-d4a5-462f-8788-27fe8de8f0f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:12:18.036433 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:12:18.108812 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd6fc6f-d4a5-462f-8788-27fe8de8f0f5 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:12:18.236843 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f25866-758e-48ed-9bca-07d47de9c0de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cd171bf4df94f8fab397e42835feeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:12:18.237383 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f25866-758e-48ed-9bca-07d47de9c0de service nova] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:18.237704 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f25866-758e-48ed-9bca-07d47de9c0de service nova] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:18.237989 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f25866-758e-48ed-9bca-07d47de9c0de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:18.239638 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-29f25866-758e-48ed-9bca-07d47de9c0de service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:12:18.282540 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:12:18.340030 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f25866-758e-48ed-9bca-07d47de9c0de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:12:18.341813 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f25866-758e-48ed-9bca-07d47de9c0de service nova] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:12:18.361148 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 696/1406] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:12:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:12:20.451312 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda2f4ab-1469-4677-9c67-c7ebfc061a5a None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:20.451884 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda2f4ab-1469-4677-9c67-c7ebfc061a5a None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:20.451884 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda2f4ab-1469-4677-9c67-c7ebfc061a5a None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:20.453220 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 711/1407] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:12:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:12:20.458777 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb8d4f4-e88a-4ffc-8d51-4e5a2d64e315 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:20.459220 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb8d4f4-e88a-4ffc-8d51-4e5a2d64e315 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:20.459314 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb8d4f4-e88a-4ffc-8d51-4e5a2d64e315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:20.528994 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-2eb8d4f4-e88a-4ffc-8d51-4e5a2d64e315 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:12:20.530232 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-2eb8d4f4-e88a-4ffc-8d51-4e5a2d64e315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:12:20.670328 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 697/1408] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:12:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:12:20.974481 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda2f4ab-1469-4677-9c67-c7ebfc061a5a None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:12:20.976970 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-bda2f4ab-1469-4677-9c67-c7ebfc061a5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:12:21.165629 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda2f4ab-1469-4677-9c67-c7ebfc061a5a None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:12:21.263760 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:12:21.327878 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c44e7d-36e5-4cd3-8264-a976731f871d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cd171bf4df94f8fab397e42835feeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:12:21.328495 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c44e7d-36e5-4cd3-8264-a976731f871d service nova] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:21.328777 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c44e7d-36e5-4cd3-8264-a976731f871d service nova] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:21.329059 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c44e7d-36e5-4cd3-8264-a976731f871d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:21.330716 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-a7c44e7d-36e5-4cd3-8264-a976731f871d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:12:21.441632 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c44e7d-36e5-4cd3-8264-a976731f871d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:12:21.446529 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c44e7d-36e5-4cd3-8264-a976731f871d service nova] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:12:21.467982 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 712/1409] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:12:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:12:30.666165 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=93 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:12:31.655784 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=65 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:12:37.095071 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f41eb03-578d-47d3-a483-c553fecaf40f None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:37.095660 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f41eb03-578d-47d3-a483-c553fecaf40f None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:37.095660 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f41eb03-578d-47d3-a483-c553fecaf40f None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:37.096931 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 698/1410] 10.222.0.58 () {54 vars in 918 bytes} [Thu Aug 25 03:12:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:12:37.102064 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e3b9e8-5eb1-4f3b-b63a-bb0c52e4abe9 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:37.102502 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e3b9e8-5eb1-4f3b-b63a-bb0c52e4abe9 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:37.102663 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e3b9e8-5eb1-4f3b-b63a-bb0c52e4abe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:37.176221 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-76e3b9e8-5eb1-4f3b-b63a-bb0c52e4abe9 None None] MFA Rules not processed for user `2b692edd3e8c41289cc42e1afb4a6dd8`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:12:37.177413 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-76e3b9e8-5eb1-4f3b-b63a-bb0c52e4abe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:12:37.280865 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 713/1411] 10.222.0.58 () {58 vars in 1015 bytes} [Thu Aug 25 03:12:37 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:12:37.295239 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445b581f-749c-4175-8147-965c7dd56c76 None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:37.295521 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445b581f-749c-4175-8147-965c7dd56c76 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:37.295796 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445b581f-749c-4175-8147-965c7dd56c76 None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:37.297113 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 699/1412] 10.222.0.58 () {54 vars in 918 bytes} [Thu Aug 25 03:12:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:12:37.301907 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2392014-aa64-411e-b296-d7ed909954a7 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:37.302192 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2392014-aa64-411e-b296-d7ed909954a7 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:37.302503 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2392014-aa64-411e-b296-d7ed909954a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:37.376188 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:12:37.378629 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-b2392014-aa64-411e-b296-d7ed909954a7 None None] MFA Rules not processed for user `0cd171bf4df94f8fab397e42835feeab`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:12:37.379825 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-b2392014-aa64-411e-b296-d7ed909954a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:12:37.494295 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 714/1413] 10.222.0.58 () {58 vars in 1015 bytes} [Thu Aug 25 03:12:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:12:39.888749 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baa9129-1cbb-4d84-ae8d-6e396f882f13 None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:39.889276 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baa9129-1cbb-4d84-ae8d-6e396f882f13 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:39.889276 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baa9129-1cbb-4d84-ae8d-6e396f882f13 None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:39.890572 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 700/1414] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 25 03:12:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:12:39.896019 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0c9443-8875-4888-a47b-63de7a471924 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:39.896480 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0c9443-8875-4888-a47b-63de7a471924 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:39.896589 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0c9443-8875-4888-a47b-63de7a471924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:39.961640 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-7a0c9443-8875-4888-a47b-63de7a471924 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:12:39.962947 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-7a0c9443-8875-4888-a47b-63de7a471924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:12:40.016547 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:12:40.058240 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 715/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 25 03:12:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:12:40.345481 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baa9129-1cbb-4d84-ae8d-6e396f882f13 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:12:40.347050 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-2baa9129-1cbb-4d84-ae8d-6e396f882f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:12:40.491311 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baa9129-1cbb-4d84-ae8d-6e396f882f13 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:12:40.563151 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:12:40.603343 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6682bf-d179-49c8-9b1e-49c571800085 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cd171bf4df94f8fab397e42835feeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:12:40.603900 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6682bf-d179-49c8-9b1e-49c571800085 service nova] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:40.604383 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6682bf-d179-49c8-9b1e-49c571800085 service nova] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:40.604827 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6682bf-d179-49c8-9b1e-49c571800085 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:40.606803 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-7c6682bf-d179-49c8-9b1e-49c571800085 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:12:40.693994 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6682bf-d179-49c8-9b1e-49c571800085 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:12:40.695648 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6682bf-d179-49c8-9b1e-49c571800085 service nova] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:12:40.713793 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 701/1416] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:12:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:12:41.625428 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e62690-ccd3-43b2-beff-b700b9886cbc None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:41.625999 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e62690-ccd3-43b2-beff-b700b9886cbc None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:41.625999 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e62690-ccd3-43b2-beff-b700b9886cbc None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:41.627438 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 716/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 25 03:12:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:12:41.632451 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd5dfbd-53cf-484d-8e89-3aeb457d96d1 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:41.633358 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd5dfbd-53cf-484d-8e89-3aeb457d96d1 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:41.633517 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd5dfbd-53cf-484d-8e89-3aeb457d96d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:41.708757 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-1cd5dfbd-53cf-484d-8e89-3aeb457d96d1 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:12:41.709963 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-1cd5dfbd-53cf-484d-8e89-3aeb457d96d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:12:41.736170 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:12:41.809913 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 702/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 25 03:12:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:12:41.960886 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e62690-ccd3-43b2-beff-b700b9886cbc None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:12:41.962421 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-a7e62690-ccd3-43b2-beff-b700b9886cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:12:42.113950 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e62690-ccd3-43b2-beff-b700b9886cbc None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:12:42.154896 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:12:42.229211 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e63d4ef-a135-4a03-b4df-d6b004e14042 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cd171bf4df94f8fab397e42835feeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:12:42.229748 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63d4ef-a135-4a03-b4df-d6b004e14042 service nova] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:42.230027 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63d4ef-a135-4a03-b4df-d6b004e14042 service nova] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:42.230322 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63d4ef-a135-4a03-b4df-d6b004e14042 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:42.231925 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-8e63d4ef-a135-4a03-b4df-d6b004e14042 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:12:42.319595 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e63d4ef-a135-4a03-b4df-d6b004e14042 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:12:42.321444 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e63d4ef-a135-4a03-b4df-d6b004e14042 service nova] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:12:42.338788 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 717/1419] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:12:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:12:43.221053 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922a35db-008e-4c51-a184-49b82245b4bd None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:43.221603 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922a35db-008e-4c51-a184-49b82245b4bd None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:43.221603 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922a35db-008e-4c51-a184-49b82245b4bd None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:43.222971 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 703/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 25 03:12:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:12:43.228381 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1d1e3f-32cd-410e-bed7-2d6874ccc9e6 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:43.228815 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1d1e3f-32cd-410e-bed7-2d6874ccc9e6 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:43.228920 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1d1e3f-32cd-410e-bed7-2d6874ccc9e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:43.281511 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:12:43.303002 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-8a1d1e3f-32cd-410e-bed7-2d6874ccc9e6 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:12:43.304136 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-8a1d1e3f-32cd-410e-bed7-2d6874ccc9e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:12:43.404217 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 718/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 25 03:12:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:12:43.559247 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922a35db-008e-4c51-a184-49b82245b4bd None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:12:43.560703 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-922a35db-008e-4c51-a184-49b82245b4bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:12:43.706455 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922a35db-008e-4c51-a184-49b82245b4bd None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:12:43.718923 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:12:43.818062 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca5431e-a4b3-466d-97bc-54faa1758bb1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cd171bf4df94f8fab397e42835feeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:12:43.818612 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca5431e-a4b3-466d-97bc-54faa1758bb1 service nova] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:43.818891 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca5431e-a4b3-466d-97bc-54faa1758bb1 service nova] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:43.819161 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca5431e-a4b3-466d-97bc-54faa1758bb1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:43.821523 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-0ca5431e-a4b3-466d-97bc-54faa1758bb1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:12:43.909077 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca5431e-a4b3-466d-97bc-54faa1758bb1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:12:43.910721 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca5431e-a4b3-466d-97bc-54faa1758bb1 service nova] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:12:43.928115 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 704/1422] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:12:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:12:44.787673 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46869381-bd58-4a98-9ce7-a5fe83271f51 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:44.788266 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46869381-bd58-4a98-9ce7-a5fe83271f51 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:44.788266 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46869381-bd58-4a98-9ce7-a5fe83271f51 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:44.790121 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 719/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 25 03:12:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:12:44.795139 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1c9488-65e7-46c2-aa93-b36776a0cd32 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:44.795579 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1c9488-65e7-46c2-aa93-b36776a0cd32 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:44.795763 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1c9488-65e7-46c2-aa93-b36776a0cd32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:44.816098 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:12:44.872468 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-be1c9488-65e7-46c2-aa93-b36776a0cd32 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:12:44.873678 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-be1c9488-65e7-46c2-aa93-b36776a0cd32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:12:44.967926 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 705/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 25 03:12:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:12:45.118052 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46869381-bd58-4a98-9ce7-a5fe83271f51 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:12:45.119652 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-46869381-bd58-4a98-9ce7-a5fe83271f51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:12:45.229216 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:12:45.264764 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46869381-bd58-4a98-9ce7-a5fe83271f51 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:12:45.375500 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f25a7b1-1e4f-4014-8e20-71dae72aba90 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cd171bf4df94f8fab397e42835feeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:12:45.376101 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f25a7b1-1e4f-4014-8e20-71dae72aba90 service nova] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:45.376379 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f25a7b1-1e4f-4014-8e20-71dae72aba90 service nova] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:45.376651 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f25a7b1-1e4f-4014-8e20-71dae72aba90 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:45.378246 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-2f25a7b1-1e4f-4014-8e20-71dae72aba90 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:12:45.451973 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:12:45.465133 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f25a7b1-1e4f-4014-8e20-71dae72aba90 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:12:45.466760 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f25a7b1-1e4f-4014-8e20-71dae72aba90 service nova] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:12:45.485296 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 720/1425] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:12:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:12:46.346571 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b526e-4cfd-4779-9a7c-b739b435adb3 None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:46.347072 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b526e-4cfd-4779-9a7c-b739b435adb3 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:46.347177 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b526e-4cfd-4779-9a7c-b739b435adb3 None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:46.348459 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 706/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 25 03:12:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:12:46.353451 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2e46b1-89dd-4289-8421-e9aff8a95db0 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:46.353897 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2e46b1-89dd-4289-8421-e9aff8a95db0 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:46.353997 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2e46b1-89dd-4289-8421-e9aff8a95db0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:46.427335 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-1b2e46b1-89dd-4289-8421-e9aff8a95db0 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:12:46.428511 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-1b2e46b1-89dd-4289-8421-e9aff8a95db0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:12:46.522677 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 721/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 25 03:12:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:12:46.671932 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536b526e-4cfd-4779-9a7c-b739b435adb3 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:12:46.673467 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-536b526e-4cfd-4779-9a7c-b739b435adb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:12:46.760367 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:12:46.824631 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536b526e-4cfd-4779-9a7c-b739b435adb3 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:12:46.937336 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22896a2-161e-4676-bb8e-3ffa12435737 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cd171bf4df94f8fab397e42835feeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:12:46.937948 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22896a2-161e-4676-bb8e-3ffa12435737 service nova] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:46.938228 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22896a2-161e-4676-bb8e-3ffa12435737 service nova] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:46.938534 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22896a2-161e-4676-bb8e-3ffa12435737 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:46.940145 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-a22896a2-161e-4676-bb8e-3ffa12435737 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:12:46.986923 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:12:47.028535 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22896a2-161e-4676-bb8e-3ffa12435737 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:12:47.030115 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22896a2-161e-4676-bb8e-3ffa12435737 service nova] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:12:47.048774 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 707/1428] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:12:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:12:47.901061 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab7b889-960f-4c55-aa71-0471bcc1de0f None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:47.901594 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab7b889-960f-4c55-aa71-0471bcc1de0f None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:47.901594 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab7b889-960f-4c55-aa71-0471bcc1de0f None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:47.902930 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 722/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 25 03:12:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:12:47.908036 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba0c421-27c4-4356-8e5e-a6d21d00b14c None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:47.908464 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba0c421-27c4-4356-8e5e-a6d21d00b14c None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:47.908574 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba0c421-27c4-4356-8e5e-a6d21d00b14c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:47.984286 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-2ba0c421-27c4-4356-8e5e-a6d21d00b14c None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:12:47.985493 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-2ba0c421-27c4-4356-8e5e-a6d21d00b14c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:12:48.080861 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 708/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 25 03:12:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:12:48.234415 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab7b889-960f-4c55-aa71-0471bcc1de0f None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:12:48.235923 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-7ab7b889-960f-4c55-aa71-0471bcc1de0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:12:48.283727 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:12:48.378644 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab7b889-960f-4c55-aa71-0471bcc1de0f None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:12:48.489974 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b796b3-1fab-4e89-91e2-784644178f30 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cd171bf4df94f8fab397e42835feeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:12:48.490550 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b796b3-1fab-4e89-91e2-784644178f30 service nova] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:48.490825 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b796b3-1fab-4e89-91e2-784644178f30 service nova] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:48.491095 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b796b3-1fab-4e89-91e2-784644178f30 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:48.492700 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-85b796b3-1fab-4e89-91e2-784644178f30 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:12:48.499611 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:12:48.582791 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b796b3-1fab-4e89-91e2-784644178f30 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:12:48.584397 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b796b3-1fab-4e89-91e2-784644178f30 service nova] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:12:48.602717 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 723/1431] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:12:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:12:49.467039 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f5d0fe-34c6-4452-8b14-2ba5d7db73c5 None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:49.467616 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f5d0fe-34c6-4452-8b14-2ba5d7db73c5 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:49.467616 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f5d0fe-34c6-4452-8b14-2ba5d7db73c5 None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:49.468918 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 709/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 25 03:12:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:12:49.474085 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c819ac87-0f91-4b7a-97d4-06bf9f03b60e None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:49.474542 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c819ac87-0f91-4b7a-97d4-06bf9f03b60e None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:49.474672 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c819ac87-0f91-4b7a-97d4-06bf9f03b60e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:49.550403 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-c819ac87-0f91-4b7a-97d4-06bf9f03b60e None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:12:49.552300 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-c819ac87-0f91-4b7a-97d4-06bf9f03b60e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:12:49.622391 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:12:49.652482 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 724/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 25 03:12:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:12:49.805039 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f5d0fe-34c6-4452-8b14-2ba5d7db73c5 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:12:49.806990 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-66f5d0fe-34c6-4452-8b14-2ba5d7db73c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:12:49.826833 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:12:49.956359 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f5d0fe-34c6-4452-8b14-2ba5d7db73c5 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:12:50.040992 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:12:50.066509 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a377f3-e778-49f3-96cf-6ebbf6dde34d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cd171bf4df94f8fab397e42835feeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:12:50.067051 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a377f3-e778-49f3-96cf-6ebbf6dde34d service nova] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:50.067544 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a377f3-e778-49f3-96cf-6ebbf6dde34d service nova] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:50.068165 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a377f3-e778-49f3-96cf-6ebbf6dde34d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:50.070203 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-51a377f3-e778-49f3-96cf-6ebbf6dde34d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:12:50.157686 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a377f3-e778-49f3-96cf-6ebbf6dde34d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:12:50.159350 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a377f3-e778-49f3-96cf-6ebbf6dde34d service nova] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:12:50.177419 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 710/1434] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:12:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:12:51.072173 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acecede-d7f4-4c54-ae9a-26aad8e47a01 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:51.072745 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acecede-d7f4-4c54-ae9a-26aad8e47a01 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:51.072745 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acecede-d7f4-4c54-ae9a-26aad8e47a01 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:51.074127 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 725/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 25 03:12:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:12:51.079292 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391adaf6-ecb0-455c-8835-cb042851c080 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:51.079696 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391adaf6-ecb0-455c-8835-cb042851c080 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:51.079969 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391adaf6-ecb0-455c-8835-cb042851c080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:51.159892 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-391adaf6-ecb0-455c-8835-cb042851c080 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:12:51.161108 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-391adaf6-ecb0-455c-8835-cb042851c080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:12:51.201987 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:12:51.255384 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 711/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 25 03:12:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:12:51.405811 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acecede-d7f4-4c54-ae9a-26aad8e47a01 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:12:51.407371 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-2acecede-d7f4-4c54-ae9a-26aad8e47a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:12:51.559563 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acecede-d7f4-4c54-ae9a-26aad8e47a01 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:12:51.613318 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:12:51.673688 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911765f6-4a8b-4144-9d90-55b764b463e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cd171bf4df94f8fab397e42835feeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:12:51.674308 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911765f6-4a8b-4144-9d90-55b764b463e9 service nova] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:51.674594 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911765f6-4a8b-4144-9d90-55b764b463e9 service nova] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:51.674862 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911765f6-4a8b-4144-9d90-55b764b463e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:51.676436 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-911765f6-4a8b-4144-9d90-55b764b463e9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:12:51.763402 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911765f6-4a8b-4144-9d90-55b764b463e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:12:51.764969 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911765f6-4a8b-4144-9d90-55b764b463e9 service nova] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:12:51.783076 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 726/1437] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:12:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:12:52.655521 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248c4fe3-034f-4c91-8361-e954561a35cf None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:52.656029 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248c4fe3-034f-4c91-8361-e954561a35cf None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:52.656140 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248c4fe3-034f-4c91-8361-e954561a35cf None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:52.657396 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 712/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 25 03:12:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:12:52.662258 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5b9140-a2e7-4e2b-9c72-045aed77eee2 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:52.662752 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5b9140-a2e7-4e2b-9c72-045aed77eee2 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:52.662869 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5b9140-a2e7-4e2b-9c72-045aed77eee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:52.738400 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-6a5b9140-a2e7-4e2b-9c72-045aed77eee2 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:12:52.739609 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-6a5b9140-a2e7-4e2b-9c72-045aed77eee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:12:52.741612 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:12:52.836869 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 727/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 25 03:12:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:12:52.986683 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248c4fe3-034f-4c91-8361-e954561a35cf None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:12:52.988197 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-248c4fe3-034f-4c91-8361-e954561a35cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:12:53.137175 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248c4fe3-034f-4c91-8361-e954561a35cf None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:12:53.163511 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:12:53.247911 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988cd34f-2053-4d24-bfd5-bdcac51faca5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cd171bf4df94f8fab397e42835feeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:12:53.248448 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988cd34f-2053-4d24-bfd5-bdcac51faca5 service nova] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:53.248731 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988cd34f-2053-4d24-bfd5-bdcac51faca5 service nova] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:53.249001 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988cd34f-2053-4d24-bfd5-bdcac51faca5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:53.250634 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-988cd34f-2053-4d24-bfd5-bdcac51faca5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:12:53.337036 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988cd34f-2053-4d24-bfd5-bdcac51faca5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:12:53.338656 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988cd34f-2053-4d24-bfd5-bdcac51faca5 service nova] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:12:53.356628 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 713/1440] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:12:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:12:55.165068 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d646cce4-328d-4092-893e-ae1aa2097370 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:55.165642 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d646cce4-328d-4092-893e-ae1aa2097370 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:55.165642 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d646cce4-328d-4092-893e-ae1aa2097370 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:55.167033 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 728/1441] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 25 03:12:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:12:55.172523 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d4b98a-4a19-4564-99b3-aef9d1303a21 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:55.172937 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d4b98a-4a19-4564-99b3-aef9d1303a21 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:55.173044 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d4b98a-4a19-4564-99b3-aef9d1303a21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:55.215062 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:12:55.238858 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-27d4b98a-4a19-4564-99b3-aef9d1303a21 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:12:55.240072 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-27d4b98a-4a19-4564-99b3-aef9d1303a21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:12:55.333545 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 714/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 25 03:12:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:12:55.622821 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d646cce4-328d-4092-893e-ae1aa2097370 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:12:55.624458 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-d646cce4-328d-4092-893e-ae1aa2097370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:12:55.728266 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d646cce4-328d-4092-893e-ae1aa2097370 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:12:55.760368 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:12:55.801274 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a46a3d-3da7-480b-b963-a893b5bdc5d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:12:55.801884 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a46a3d-3da7-480b-b963-a893b5bdc5d4 service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:55.802166 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a46a3d-3da7-480b-b963-a893b5bdc5d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:55.802474 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a46a3d-3da7-480b-b963-a893b5bdc5d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:55.804074 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-27a46a3d-3da7-480b-b963-a893b5bdc5d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:12:55.912492 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a46a3d-3da7-480b-b963-a893b5bdc5d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:12:55.914497 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a46a3d-3da7-480b-b963-a893b5bdc5d4 service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:12:55.933160 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 729/1443] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:12:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:12:58.009146 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c1c1cc-39b2-4b65-bc36-22193582fabe None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:58.018140 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c1c1cc-39b2-4b65-bc36-22193582fabe None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:58.018140 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c1c1cc-39b2-4b65-bc36-22193582fabe None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:58.018140 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 715/1444] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 25 03:12:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:12:58.018524 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c9c2a9-2b42-4f14-8b1b-ad30a5af6cc1 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:58.018524 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c9c2a9-2b42-4f14-8b1b-ad30a5af6cc1 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:58.018524 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c9c2a9-2b42-4f14-8b1b-ad30a5af6cc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:58.083145 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-25c9c2a9-2b42-4f14-8b1b-ad30a5af6cc1 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:12:58.084356 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-25c9c2a9-2b42-4f14-8b1b-ad30a5af6cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:12:58.097730 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:12:58.179975 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 730/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 25 03:12:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:12:58.465876 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c1c1cc-39b2-4b65-bc36-22193582fabe None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:12:58.467442 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-41c1c1cc-39b2-4b65-bc36-22193582fabe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:12:58.568666 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c1c1cc-39b2-4b65-bc36-22193582fabe None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:12:58.581458 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:12:58.643130 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fed816-8202-448d-886a-57b3da4e5656 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:12:58.643747 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fed816-8202-448d-886a-57b3da4e5656 service neutron] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:58.644033 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fed816-8202-448d-886a-57b3da4e5656 service neutron] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:58.644303 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fed816-8202-448d-886a-57b3da4e5656 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:58.645994 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-71fed816-8202-448d-886a-57b3da4e5656 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:12:58.752256 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fed816-8202-448d-886a-57b3da4e5656 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:12:58.753613 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fed816-8202-448d-886a-57b3da4e5656 service neutron] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:12:58.771003 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 716/1446] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:12:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:13:01.074492 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad425941-67eb-49b5-9f3f-b755c0ef53c0 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:01.075032 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad425941-67eb-49b5-9f3f-b755c0ef53c0 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:01.075445 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad425941-67eb-49b5-9f3f-b755c0ef53c0 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:01.076887 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 731/1447] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 25 03:13:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:13:01.082363 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccd3f2c-007e-436d-9cd5-9d48d93cd8fc None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:01.082772 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccd3f2c-007e-436d-9cd5-9d48d93cd8fc None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:01.082924 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccd3f2c-007e-436d-9cd5-9d48d93cd8fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:01.132477 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:13:01.151179 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-dccd3f2c-007e-436d-9cd5-9d48d93cd8fc None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:13:01.152391 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-dccd3f2c-007e-436d-9cd5-9d48d93cd8fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:13:01.247596 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 717/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 25 03:13:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:13:01.535646 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad425941-67eb-49b5-9f3f-b755c0ef53c0 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:13:01.537237 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-ad425941-67eb-49b5-9f3f-b755c0ef53c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:13:01.641626 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad425941-67eb-49b5-9f3f-b755c0ef53c0 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:13:01.683563 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:13:01.717435 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ac3e43-4f9a-4c01-99ad-e1490b13f53f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:13:01.718047 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ac3e43-4f9a-4c01-99ad-e1490b13f53f service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:01.718401 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ac3e43-4f9a-4c01-99ad-e1490b13f53f service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:01.718728 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ac3e43-4f9a-4c01-99ad-e1490b13f53f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:01.720319 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-44ac3e43-4f9a-4c01-99ad-e1490b13f53f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:13:01.829095 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ac3e43-4f9a-4c01-99ad-e1490b13f53f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:13:01.830502 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ac3e43-4f9a-4c01-99ad-e1490b13f53f service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:13:01.850298 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 732/1449] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:13:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:13:03.894470 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f83a60-b92d-4451-b813-a6b79944e14e None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:03.895020 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f83a60-b92d-4451-b813-a6b79944e14e None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:03.895020 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f83a60-b92d-4451-b813-a6b79944e14e None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:03.896272 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 718/1450] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 25 03:13:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:13:03.901636 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c0bbee-dcd8-4590-b22f-d2e43fea1458 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:03.902077 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c0bbee-dcd8-4590-b22f-d2e43fea1458 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:03.902177 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c0bbee-dcd8-4590-b22f-d2e43fea1458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:03.966751 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-a8c0bbee-dcd8-4590-b22f-d2e43fea1458 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:13:03.967933 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-a8c0bbee-dcd8-4590-b22f-d2e43fea1458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:13:03.988182 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:13:04.064101 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 733/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 25 03:13:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:13:04.346972 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f83a60-b92d-4451-b813-a6b79944e14e None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:13:04.348508 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-86f83a60-b92d-4451-b813-a6b79944e14e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:13:04.449041 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f83a60-b92d-4451-b813-a6b79944e14e None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:13:04.468707 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:13:04.520592 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c79939-4aaa-4a6b-acdb-198a59598774 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:13:04.521196 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c79939-4aaa-4a6b-acdb-198a59598774 service neutron] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:04.521475 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c79939-4aaa-4a6b-acdb-198a59598774 service neutron] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:04.521742 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c79939-4aaa-4a6b-acdb-198a59598774 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:04.523931 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-b3c79939-4aaa-4a6b-acdb-198a59598774 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:13:04.632439 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c79939-4aaa-4a6b-acdb-198a59598774 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:13:04.633984 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c79939-4aaa-4a6b-acdb-198a59598774 service neutron] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:13:04.652779 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 719/1452] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:13:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:13:06.632107 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445e5147-03a7-4927-88c9-c077f86a656a None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:06.633356 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445e5147-03a7-4927-88c9-c077f86a656a None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:06.633356 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445e5147-03a7-4927-88c9-c077f86a656a None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:06.634676 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 734/1453] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 25 03:13:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:13:06.640291 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbd552d-4d84-404c-a54f-b80c87931dd4 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:06.640676 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbd552d-4d84-404c-a54f-b80c87931dd4 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:06.640845 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbd552d-4d84-404c-a54f-b80c87931dd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:06.706678 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:13:06.710594 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-0fbd552d-4d84-404c-a54f-b80c87931dd4 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:13:06.711813 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-0fbd552d-4d84-404c-a54f-b80c87931dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:13:06.807663 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 720/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 25 03:13:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:13:07.105214 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445e5147-03a7-4927-88c9-c077f86a656a None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:13:07.106834 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-445e5147-03a7-4927-88c9-c077f86a656a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:13:07.211721 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445e5147-03a7-4927-88c9-c077f86a656a None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:13:07.260323 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:13:07.284442 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae34846-3f97-4669-b4c0-dd0f39aec521 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:13:07.285042 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae34846-3f97-4669-b4c0-dd0f39aec521 service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:07.285348 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae34846-3f97-4669-b4c0-dd0f39aec521 service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:07.285621 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae34846-3f97-4669-b4c0-dd0f39aec521 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:07.287253 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-9ae34846-3f97-4669-b4c0-dd0f39aec521 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:13:07.395521 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae34846-3f97-4669-b4c0-dd0f39aec521 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:13:07.396879 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae34846-3f97-4669-b4c0-dd0f39aec521 service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:13:07.416969 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 735/1455] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:13:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:13:09.394645 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19af7123-d5b4-45b1-bbec-4db8e1b09ef1 None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:09.395227 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19af7123-d5b4-45b1-bbec-4db8e1b09ef1 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:09.395227 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19af7123-d5b4-45b1-bbec-4db8e1b09ef1 None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:09.396540 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 721/1456] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 25 03:13:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:13:09.402751 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ea0a8f-ff1f-4e0b-9f1f-427df8921421 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:09.403205 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ea0a8f-ff1f-4e0b-9f1f-427df8921421 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:09.403388 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ea0a8f-ff1f-4e0b-9f1f-427df8921421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:09.468271 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-b5ea0a8f-ff1f-4e0b-9f1f-427df8921421 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:13:09.469475 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-b5ea0a8f-ff1f-4e0b-9f1f-427df8921421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:13:09.497793 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:13:09.564526 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 736/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 25 03:13:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:13:09.850980 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19af7123-d5b4-45b1-bbec-4db8e1b09ef1 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:13:09.852524 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-19af7123-d5b4-45b1-bbec-4db8e1b09ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:13:09.956149 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19af7123-d5b4-45b1-bbec-4db8e1b09ef1 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:13:09.983588 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:13:10.027624 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bb3297-519d-4b17-88f3-0601e889a590 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:13:10.028227 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bb3297-519d-4b17-88f3-0601e889a590 service neutron] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:10.028504 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bb3297-519d-4b17-88f3-0601e889a590 service neutron] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:10.028774 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bb3297-519d-4b17-88f3-0601e889a590 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:10.030388 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-42bb3297-519d-4b17-88f3-0601e889a590 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:13:10.136664 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bb3297-519d-4b17-88f3-0601e889a590 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:13:10.138134 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bb3297-519d-4b17-88f3-0601e889a590 service neutron] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:13:10.156199 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 722/1458] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:13:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:13:12.126001 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8ff32c-f751-444a-b398-b6af5d9b6334 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:12.126566 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8ff32c-f751-444a-b398-b6af5d9b6334 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:12.126566 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8ff32c-f751-444a-b398-b6af5d9b6334 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:12.127921 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 737/1459] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 25 03:13:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:13:12.133805 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928466a8-e727-4a8b-b797-c87dae455aec None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:12.134236 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928466a8-e727-4a8b-b797-c87dae455aec None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:12.134418 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928466a8-e727-4a8b-b797-c87dae455aec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:12.199156 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-928466a8-e727-4a8b-b797-c87dae455aec None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:13:12.200368 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-928466a8-e727-4a8b-b797-c87dae455aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:13:12.208604 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:13:12.307698 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 723/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 25 03:13:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:13:12.590760 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8ff32c-f751-444a-b398-b6af5d9b6334 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:13:12.592281 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-ef8ff32c-f751-444a-b398-b6af5d9b6334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:13:12.696393 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8ff32c-f751-444a-b398-b6af5d9b6334 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:13:12.751226 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:13:12.770594 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a308eb8d-1902-441a-8938-f5b2d1b861ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:13:12.771208 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a308eb8d-1902-441a-8938-f5b2d1b861ca service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:12.771492 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a308eb8d-1902-441a-8938-f5b2d1b861ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:12.771768 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a308eb8d-1902-441a-8938-f5b2d1b861ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:12.773413 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-a308eb8d-1902-441a-8938-f5b2d1b861ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:13:12.885770 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a308eb8d-1902-441a-8938-f5b2d1b861ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:13:12.887191 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a308eb8d-1902-441a-8938-f5b2d1b861ca service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:13:12.908196 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 738/1461] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:13:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:13:14.877506 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d574771-d217-4ec0-92e3-6bfeac89cfbd None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:14.878059 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d574771-d217-4ec0-92e3-6bfeac89cfbd None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:14.878059 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d574771-d217-4ec0-92e3-6bfeac89cfbd None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:14.879407 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 724/1462] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 25 03:13:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:13:14.884797 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9f3883-a526-408b-b378-46067a010b3f None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:14.885238 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9f3883-a526-408b-b378-46067a010b3f None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:14.885343 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9f3883-a526-408b-b378-46067a010b3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:14.952552 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-ff9f3883-a526-408b-b378-46067a010b3f None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:13:14.953779 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-ff9f3883-a526-408b-b378-46067a010b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:13:14.991893 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:13:15.048853 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 739/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 25 03:13:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:13:15.339559 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d574771-d217-4ec0-92e3-6bfeac89cfbd None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:13:15.341090 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-1d574771-d217-4ec0-92e3-6bfeac89cfbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:13:15.441211 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d574771-d217-4ec0-92e3-6bfeac89cfbd None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:13:15.479072 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:13:15.514910 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c469ea9-178b-4d5a-81f5-d4da2c6bd291 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:13:15.515515 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c469ea9-178b-4d5a-81f5-d4da2c6bd291 service neutron] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:15.515793 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c469ea9-178b-4d5a-81f5-d4da2c6bd291 service neutron] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:15.516062 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c469ea9-178b-4d5a-81f5-d4da2c6bd291 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:15.517639 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-9c469ea9-178b-4d5a-81f5-d4da2c6bd291 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:13:15.623390 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c469ea9-178b-4d5a-81f5-d4da2c6bd291 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:13:15.624751 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c469ea9-178b-4d5a-81f5-d4da2c6bd291 service neutron] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:13:15.643705 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 725/1464] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:13:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:13:17.573256 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58729361-6bdc-4119-a573-af49e2234eb1 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:17.573843 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58729361-6bdc-4119-a573-af49e2234eb1 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:17.573843 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58729361-6bdc-4119-a573-af49e2234eb1 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:17.575200 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 740/1465] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 25 03:13:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:13:17.580813 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e138f275-023e-4bf1-877e-9b153a499f5c None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:17.581264 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e138f275-023e-4bf1-877e-9b153a499f5c None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:17.581363 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e138f275-023e-4bf1-877e-9b153a499f5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:17.646027 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-e138f275-023e-4bf1-877e-9b153a499f5c None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:13:17.647268 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-e138f275-023e-4bf1-877e-9b153a499f5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:13:17.666103 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:13:17.744945 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 726/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 25 03:13:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:13:18.029021 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58729361-6bdc-4119-a573-af49e2234eb1 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:13:18.030608 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-58729361-6bdc-4119-a573-af49e2234eb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:13:18.136212 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58729361-6bdc-4119-a573-af49e2234eb1 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:13:18.205020 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:13:18.212356 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56227f6-bc53-4e37-8ebb-5872121c7881 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:13:18.212967 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56227f6-bc53-4e37-8ebb-5872121c7881 service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:18.213247 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56227f6-bc53-4e37-8ebb-5872121c7881 service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:18.213520 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56227f6-bc53-4e37-8ebb-5872121c7881 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:18.215183 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-a56227f6-bc53-4e37-8ebb-5872121c7881 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:13:18.322835 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56227f6-bc53-4e37-8ebb-5872121c7881 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:13:18.324185 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56227f6-bc53-4e37-8ebb-5872121c7881 service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:13:18.342366 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 741/1467] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:13:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 25 03:13:20.258501 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2222b8c-6a73-4afc-b861-bec2405af509 None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:20.259071 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2222b8c-6a73-4afc-b861-bec2405af509 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:20.259071 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2222b8c-6a73-4afc-b861-bec2405af509 None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:20.260383 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 727/1468] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 25 03:13:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:13:20.265736 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728dd7a7-b854-45ba-84ab-0dea4655ee5f None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:20.266178 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728dd7a7-b854-45ba-84ab-0dea4655ee5f None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:20.266357 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728dd7a7-b854-45ba-84ab-0dea4655ee5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:20.331212 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-728dd7a7-b854-45ba-84ab-0dea4655ee5f None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:13:20.332403 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-728dd7a7-b854-45ba-84ab-0dea4655ee5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:13:20.380959 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:13:20.428428 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 742/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 25 03:13:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:13:20.720194 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2222b8c-6a73-4afc-b861-bec2405af509 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:13:20.721699 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-a2222b8c-6a73-4afc-b861-bec2405af509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:13:20.822064 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2222b8c-6a73-4afc-b861-bec2405af509 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:13:20.869246 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:13:20.893658 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12847314-3899-4746-9f77-8a3eed620d89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:13:20.894204 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12847314-3899-4746-9f77-8a3eed620d89 service neutron] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:20.894512 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12847314-3899-4746-9f77-8a3eed620d89 service neutron] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:20.894786 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12847314-3899-4746-9f77-8a3eed620d89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:20.896396 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-12847314-3899-4746-9f77-8a3eed620d89 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:13:21.002130 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12847314-3899-4746-9f77-8a3eed620d89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:13:21.003589 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12847314-3899-4746-9f77-8a3eed620d89 service neutron] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:13:21.022003 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 728/1470] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:13:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:13:30.426682 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=22 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:13:31.019674 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=66 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:13:41.622064 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be83e7d-aa8a-4beb-8087-564527c940e2 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:41.622684 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be83e7d-aa8a-4beb-8087-564527c940e2 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:41.622684 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be83e7d-aa8a-4beb-8087-564527c940e2 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:41.624001 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 743/1471] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:13:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:13:41.629240 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f997a2c4-f063-4a6e-84a5-61fba918cfde None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:41.629700 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f997a2c4-f063-4a6e-84a5-61fba918cfde None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:41.629802 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f997a2c4-f063-4a6e-84a5-61fba918cfde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:41.697439 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-f997a2c4-f063-4a6e-84a5-61fba918cfde None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:13:41.698711 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-f997a2c4-f063-4a6e-84a5-61fba918cfde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:13:41.793165 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 729/1472] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:13:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:13:42.093750 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7a0b3d-9f14-4fff-a8e9-6e73c3061d03 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:42.094045 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7a0b3d-9f14-4fff-a8e9-6e73c3061d03 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:42.094395 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7a0b3d-9f14-4fff-a8e9-6e73c3061d03 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:42.095819 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 744/1473] 10.222.0.58 () {54 vars in 967 bytes} [Thu Aug 25 03:13:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 25 03:13:42.102072 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9115fb8b-c4cb-442c-a6db-5edcf7fc6c46 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:42.102507 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9115fb8b-c4cb-442c-a6db-5edcf7fc6c46 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:42.102896 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9115fb8b-c4cb-442c-a6db-5edcf7fc6c46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:42.178835 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-9115fb8b-c4cb-442c-a6db-5edcf7fc6c46 None None] MFA Rules not processed for user `3f89089b390c4b6599e54000c0fa082a`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:13:42.180076 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-9115fb8b-c4cb-442c-a6db-5edcf7fc6c46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:13:42.200775 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:13:42.254832 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 730/1474] 10.222.0.58 () {58 vars in 1064 bytes} [Thu Aug 25 03:13:42 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:13:42.262960 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7a0b3d-9f14-4fff-a8e9-6e73c3061d03 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:13:42.266552 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-fd7a0b3d-9f14-4fff-a8e9-6e73c3061d03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:13:42.366440 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7a0b3d-9f14-4fff-a8e9-6e73c3061d03 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:13:42.442194 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fffad32-83c3-47ea-a934-925565da47dd service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f89089b390c4b6599e54000c0fa082a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:13:42.442834 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fffad32-83c3-47ea-a934-925565da47dd service glance] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:42.443114 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fffad32-83c3-47ea-a934-925565da47dd service glance] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:42.443384 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fffad32-83c3-47ea-a934-925565da47dd service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:42.445503 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-3fffad32-83c3-47ea-a934-925565da47dd service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:13:42.499236 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:13:42.553726 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fffad32-83c3-47ea-a934-925565da47dd service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:13:42.555253 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fffad32-83c3-47ea-a934-925565da47dd service glance] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:13:42.573907 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 745/1475] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:13:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:13:44.595193 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089fecc9-fc80-4997-ae16-0797e940a661 None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:44.595770 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089fecc9-fc80-4997-ae16-0797e940a661 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:44.595888 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089fecc9-fc80-4997-ae16-0797e940a661 None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:44.597179 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 731/1476] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:13:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:13:44.602954 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c912f1-fb0b-4cf8-a2d2-3f433d82049c None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:44.603376 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c912f1-fb0b-4cf8-a2d2-3f433d82049c None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:44.603533 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c912f1-fb0b-4cf8-a2d2-3f433d82049c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:44.669492 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-c4c912f1-fb0b-4cf8-a2d2-3f433d82049c None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:13:44.670732 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-c4c912f1-fb0b-4cf8-a2d2-3f433d82049c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:13:44.766526 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 746/1477] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:13:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:13:45.041789 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089fecc9-fc80-4997-ae16-0797e940a661 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:13:45.043393 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-089fecc9-fc80-4997-ae16-0797e940a661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:13:45.170036 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089fecc9-fc80-4997-ae16-0797e940a661 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:13:45.217820 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:13:45.261293 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9c1534-4518-430e-b9f9-044ffe0cdda1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:13:45.261895 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9c1534-4518-430e-b9f9-044ffe0cdda1 admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:45.262174 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9c1534-4518-430e-b9f9-044ffe0cdda1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:45.262483 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9c1534-4518-430e-b9f9-044ffe0cdda1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:45.263124 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9c1534-4518-430e-b9f9-044ffe0cdda1 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:13:45.264861 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: WARNING py.warnings [None req-dc9c1534-4518-430e-b9f9-044ffe0cdda1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 25 03:13:45.264861 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: warnings.warn(msg) Aug 25 03:13:45.264861 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: Aug 25 03:13:45.266059 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9c1534-4518-430e-b9f9-044ffe0cdda1 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:13:45.285934 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 732/1478] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Aug 25 03:13:45 2022] GET /identity/v3/projects => generated 2553 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:13:47.187018 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5b3f3d-d7a4-4f2f-b90f-306025e705e9 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:47.187623 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5b3f3d-d7a4-4f2f-b90f-306025e705e9 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:47.187623 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5b3f3d-d7a4-4f2f-b90f-306025e705e9 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:47.188888 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 747/1479] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:13:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:13:47.194118 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de023395-7a91-4545-b82e-3de718214202 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:47.194569 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de023395-7a91-4545-b82e-3de718214202 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:47.195332 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de023395-7a91-4545-b82e-3de718214202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:47.261692 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-de023395-7a91-4545-b82e-3de718214202 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:13:47.262930 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-de023395-7a91-4545-b82e-3de718214202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:13:47.360560 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 733/1480] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:13:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:13:47.655212 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5b3f3d-d7a4-4f2f-b90f-306025e705e9 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:13:47.656766 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-cf5b3f3d-d7a4-4f2f-b90f-306025e705e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:13:47.666545 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:13:47.801626 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5b3f3d-d7a4-4f2f-b90f-306025e705e9 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:13:47.880474 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:13:47.912916 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb10a6f-b451-4b79-b38b-fe69936241e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cd171bf4df94f8fab397e42835feeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:13:47.913555 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb10a6f-b451-4b79-b38b-fe69936241e1 service nova] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:47.913835 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb10a6f-b451-4b79-b38b-fe69936241e1 service nova] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:47.914113 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb10a6f-b451-4b79-b38b-fe69936241e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:47.915751 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-beb10a6f-b451-4b79-b38b-fe69936241e1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:13:48.003301 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb10a6f-b451-4b79-b38b-fe69936241e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:13:48.004967 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb10a6f-b451-4b79-b38b-fe69936241e1 service nova] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:13:48.023925 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 748/1481] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:13:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:13:50.132118 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc11b0a-7bb0-4aaf-b5f9-88ae2be7669a None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:50.132683 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc11b0a-7bb0-4aaf-b5f9-88ae2be7669a None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:50.132683 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc11b0a-7bb0-4aaf-b5f9-88ae2be7669a None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:50.134043 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 734/1482] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:13:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:13:50.139280 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c6262b-4283-4814-b4d6-02fa21383f6f None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:50.139680 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c6262b-4283-4814-b4d6-02fa21383f6f None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:50.139822 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c6262b-4283-4814-b4d6-02fa21383f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:50.205404 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-51c6262b-4283-4814-b4d6-02fa21383f6f None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:13:50.206645 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-51c6262b-4283-4814-b4d6-02fa21383f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:13:50.253074 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:13:50.305677 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 749/1483] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:13:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:13:50.598732 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc11b0a-7bb0-4aaf-b5f9-88ae2be7669a None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:13:50.600284 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-3cc11b0a-7bb0-4aaf-b5f9-88ae2be7669a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:13:50.623068 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:13:50.702881 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc11b0a-7bb0-4aaf-b5f9-88ae2be7669a None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:13:50.778646 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26554dec-6f4b-45f7-835c-93ffe284694e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:13:50.779259 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26554dec-6f4b-45f7-835c-93ffe284694e service neutron] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:50.779541 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26554dec-6f4b-45f7-835c-93ffe284694e service neutron] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:50.779817 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26554dec-6f4b-45f7-835c-93ffe284694e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:50.781424 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-26554dec-6f4b-45f7-835c-93ffe284694e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:13:50.853892 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:13:50.888771 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26554dec-6f4b-45f7-835c-93ffe284694e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:13:50.890207 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26554dec-6f4b-45f7-835c-93ffe284694e service neutron] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:13:50.908310 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 735/1484] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:13:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:13:52.892315 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa66059-7bf4-4ef4-b898-5dc830e07ea6 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:52.892847 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa66059-7bf4-4ef4-b898-5dc830e07ea6 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:52.892847 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa66059-7bf4-4ef4-b898-5dc830e07ea6 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:52.894128 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 750/1485] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:13:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:13:52.899458 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78717b1a-cfc1-4e2f-99a4-7a961503dda2 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:52.899860 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78717b1a-cfc1-4e2f-99a4-7a961503dda2 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:52.900010 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78717b1a-cfc1-4e2f-99a4-7a961503dda2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:52.964481 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-78717b1a-cfc1-4e2f-99a4-7a961503dda2 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:13:52.965690 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-78717b1a-cfc1-4e2f-99a4-7a961503dda2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:13:53.065418 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 736/1486] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:13:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:13:53.363917 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa66059-7bf4-4ef4-b898-5dc830e07ea6 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:13:53.365483 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-afa66059-7bf4-4ef4-b898-5dc830e07ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:13:53.471922 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa66059-7bf4-4ef4-b898-5dc830e07ea6 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:13:53.546392 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:13:53.548340 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a95d14-3592-4eaf-8c03-5d0c169baeef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:13:53.548969 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a95d14-3592-4eaf-8c03-5d0c169baeef service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:53.549292 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a95d14-3592-4eaf-8c03-5d0c169baeef service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:53.549578 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a95d14-3592-4eaf-8c03-5d0c169baeef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:53.551379 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-67a95d14-3592-4eaf-8c03-5d0c169baeef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:13:53.664772 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a95d14-3592-4eaf-8c03-5d0c169baeef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:13:53.666374 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a95d14-3592-4eaf-8c03-5d0c169baeef service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:13:53.686418 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 751/1487] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:13:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:13:55.691820 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dc9fc8-bf7f-4aec-a540-eda9ae196eab None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:55.692274 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dc9fc8-bf7f-4aec-a540-eda9ae196eab None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:55.692374 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dc9fc8-bf7f-4aec-a540-eda9ae196eab None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:55.693697 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 737/1488] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:13:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:13:55.699914 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14be219c-d249-4902-a412-d4e511a79b39 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:55.700315 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14be219c-d249-4902-a412-d4e511a79b39 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:55.700465 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14be219c-d249-4902-a412-d4e511a79b39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:55.767594 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-14be219c-d249-4902-a412-d4e511a79b39 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:13:55.768834 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-14be219c-d249-4902-a412-d4e511a79b39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:13:55.821231 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:13:55.864087 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 752/1489] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:13:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:13:56.154263 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dc9fc8-bf7f-4aec-a540-eda9ae196eab None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:13:56.155854 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-31dc9fc8-bf7f-4aec-a540-eda9ae196eab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:13:56.193086 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:13:56.258650 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dc9fc8-bf7f-4aec-a540-eda9ae196eab None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:13:56.330831 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eaf1b3-d294-4ce8-9a15-7d7eabb1c3a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:13:56.331434 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eaf1b3-d294-4ce8-9a15-7d7eabb1c3a7 service neutron] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:56.331713 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eaf1b3-d294-4ce8-9a15-7d7eabb1c3a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:56.331986 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eaf1b3-d294-4ce8-9a15-7d7eabb1c3a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:56.333575 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-c2eaf1b3-d294-4ce8-9a15-7d7eabb1c3a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:13:56.412022 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:13:56.439875 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2eaf1b3-d294-4ce8-9a15-7d7eabb1c3a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:13:56.441311 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2eaf1b3-d294-4ce8-9a15-7d7eabb1c3a7 service neutron] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:13:56.459696 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 738/1490] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:13:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:13:58.722020 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c0c572-1735-4cfd-8735-22c4481de08e None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:58.722610 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c0c572-1735-4cfd-8735-22c4481de08e None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:58.722720 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c0c572-1735-4cfd-8735-22c4481de08e None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:58.724023 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 753/1491] 10.222.0.58 () {54 vars in 927 bytes} [Thu Aug 25 03:13:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:13:58.729777 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4f6e37-3994-4d3d-a052-760f70980056 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:58.730223 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4f6e37-3994-4d3d-a052-760f70980056 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:58.730358 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4f6e37-3994-4d3d-a052-760f70980056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:58.812325 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-2d4f6e37-3994-4d3d-a052-760f70980056 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:13:58.813557 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-2d4f6e37-3994-4d3d-a052-760f70980056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:13:58.910538 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 739/1492] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Aug 25 03:13:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:13:59.205945 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c0c572-1735-4cfd-8735-22c4481de08e None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:13:59.207518 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-57c0c572-1735-4cfd-8735-22c4481de08e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:13:59.310849 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c0c572-1735-4cfd-8735-22c4481de08e None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:13:59.382763 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40e7cff-6dff-44e9-9370-680f961f7d21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:13:59.383332 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40e7cff-6dff-44e9-9370-680f961f7d21 service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:59.383609 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40e7cff-6dff-44e9-9370-680f961f7d21 service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:59.383877 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40e7cff-6dff-44e9-9370-680f961f7d21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:59.385453 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-f40e7cff-6dff-44e9-9370-680f961f7d21 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:13:59.394371 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:13:59.496033 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40e7cff-6dff-44e9-9370-680f961f7d21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:13:59.497429 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40e7cff-6dff-44e9-9370-680f961f7d21 service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:13:59.514934 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 754/1493] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:13:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:14:08.908726 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=82 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:14:09.512702 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=50 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:15:18.665702 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ce0eb0-0a83-40dc-85e7-83e72c85a523 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:15:18.666184 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ce0eb0-0a83-40dc-85e7-83e72c85a523 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:15:18.666322 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ce0eb0-0a83-40dc-85e7-83e72c85a523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:15:18.739425 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-38ce0eb0-0a83-40dc-85e7-83e72c85a523 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:15:18.740625 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-38ce0eb0-0a83-40dc-85e7-83e72c85a523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:15:18.834903 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 740/1494] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:15:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:15:18.842933 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1864ad5d-4788-480a-bfcd-749bbd2067da None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:15:18.843418 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1864ad5d-4788-480a-bfcd-749bbd2067da None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:15:18.843525 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1864ad5d-4788-480a-bfcd-749bbd2067da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:15:18.917613 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-1864ad5d-4788-480a-bfcd-749bbd2067da None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:15:18.918838 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-1864ad5d-4788-480a-bfcd-749bbd2067da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:15:19.015452 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 755/1495] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:15:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:15:19.019922 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ce0eb0-0a83-40dc-85e7-83e72c85a523 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:15:19.021117 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-38ce0eb0-0a83-40dc-85e7-83e72c85a523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:15:19.032685 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ce0eb0-0a83-40dc-85e7-83e72c85a523 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:15:19.092569 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:15:19.125693 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5436b9cb-6970-4b0a-9ded-1a98e882a34b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:15:19.126329 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5436b9cb-6970-4b0a-9ded-1a98e882a34b admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:15:19.126620 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5436b9cb-6970-4b0a-9ded-1a98e882a34b admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:15:19.126896 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5436b9cb-6970-4b0a-9ded-1a98e882a34b admin admin] PATH_INFO: `/v3/domains` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:15:19.127543 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5436b9cb-6970-4b0a-9ded-1a98e882a34b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:15:19.127847 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5436b9cb-6970-4b0a-9ded-1a98e882a34b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:15:19.128865 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: WARNING py.warnings [None req-5436b9cb-6970-4b0a-9ded-1a98e882a34b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 25 03:15:19.128865 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: warnings.warn(msg) Aug 25 03:15:19.128865 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: Aug 25 03:15:19.129761 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5436b9cb-6970-4b0a-9ded-1a98e882a34b admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:15:19.146858 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 741/1496] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Aug 25 03:15:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:15:19.149041 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1864ad5d-4788-480a-bfcd-749bbd2067da None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:15:19.158091 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1864ad5d-4788-480a-bfcd-749bbd2067da None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:15:19.218268 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:15:19.251809 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88606385-9b4d-416a-973f-f9a03282f9d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:15:19.252415 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88606385-9b4d-416a-973f-f9a03282f9d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:15:19.252697 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88606385-9b4d-416a-973f-f9a03282f9d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:15:19.252975 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88606385-9b4d-416a-973f-f9a03282f9d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:15:19.253594 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88606385-9b4d-416a-973f-f9a03282f9d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:15:19.255343 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: WARNING py.warnings [None req-88606385-9b4d-416a-973f-f9a03282f9d3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 25 03:15:19.255343 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: warnings.warn(msg) Aug 25 03:15:19.255343 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: Aug 25 03:15:19.256566 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88606385-9b4d-416a-973f-f9a03282f9d3 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:15:19.405326 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 756/1497] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:15:19 2022] POST /identity/v3/projects => generated 379 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:15:19.409545 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5436b9cb-6970-4b0a-9ded-1a98e882a34b admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:15:19.411107 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-5436b9cb-6970-4b0a-9ded-1a98e882a34b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:15:19.443920 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5436b9cb-6970-4b0a-9ded-1a98e882a34b admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:15:19.535752 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac43342-c159-4224-b510-cb263cd6b06f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:15:19.536365 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac43342-c159-4224-b510-cb263cd6b06f admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:15:19.536664 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac43342-c159-4224-b510-cb263cd6b06f admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:15:19.536935 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac43342-c159-4224-b510-cb263cd6b06f admin admin] PATH_INFO: `/v3/users` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:15:19.537552 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac43342-c159-4224-b510-cb263cd6b06f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:15:19.538580 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: WARNING py.warnings [None req-7ac43342-c159-4224-b510-cb263cd6b06f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 25 03:15:19.538580 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: warnings.warn(msg) Aug 25 03:15:19.538580 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: Aug 25 03:15:19.540466 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac43342-c159-4224-b510-cb263cd6b06f admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:15:19.642098 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 742/1498] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:15:19 2022] POST /identity/v3/users => generated 290 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:15:19.646388 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88606385-9b4d-416a-973f-f9a03282f9d3 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:15:19.647889 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-88606385-9b4d-416a-973f-f9a03282f9d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:15:19.710536 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88606385-9b4d-416a-973f-f9a03282f9d3 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:15:19.794003 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:15:19.805203 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c440b375-1cb4-408a-8158-7bb9d53bb077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:15:19.806780 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c440b375-1cb4-408a-8158-7bb9d53bb077 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:15:19.806780 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c440b375-1cb4-408a-8158-7bb9d53bb077 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:15:19.806780 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c440b375-1cb4-408a-8158-7bb9d53bb077 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:15:19.807220 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c440b375-1cb4-408a-8158-7bb9d53bb077 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:15:19.808445 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: WARNING py.warnings [None req-c440b375-1cb4-408a-8158-7bb9d53bb077 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 25 03:15:19.808445 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: warnings.warn(msg) Aug 25 03:15:19.808445 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: Aug 25 03:15:19.810960 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c440b375-1cb4-408a-8158-7bb9d53bb077 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:15:19.825259 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 757/1499] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:15:19 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 25 03:15:19.829319 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac43342-c159-4224-b510-cb263cd6b06f admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:15:19.835668 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-7ac43342-c159-4224-b510-cb263cd6b06f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:15:19.842714 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac43342-c159-4224-b510-cb263cd6b06f admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:15:19.868846 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:15:19.938097 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4121fe96-86e5-4b77-8923-30240036ebd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:15:19.938659 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4121fe96-86e5-4b77-8923-30240036ebd8 admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:15:19.938935 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4121fe96-86e5-4b77-8923-30240036ebd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:15:19.939203 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4121fe96-86e5-4b77-8923-30240036ebd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:15:19.939839 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4121fe96-86e5-4b77-8923-30240036ebd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:15:19.941056 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: WARNING py.warnings [None req-4121fe96-86e5-4b77-8923-30240036ebd8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 25 03:15:19.941056 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: warnings.warn(msg) Aug 25 03:15:19.941056 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: Aug 25 03:15:19.941935 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4121fe96-86e5-4b77-8923-30240036ebd8 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:15:19.954709 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 743/1500] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:15:19 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:15:19.958971 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c440b375-1cb4-408a-8158-7bb9d53bb077 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:15:19.960525 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-c440b375-1cb4-408a-8158-7bb9d53bb077 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:15:19.995642 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c440b375-1cb4-408a-8158-7bb9d53bb077 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:15:20.088500 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17cf987-21c4-4605-ae7a-682e699d45fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:15:20.089161 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17cf987-21c4-4605-ae7a-682e699d45fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:15:20.089445 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17cf987-21c4-4605-ae7a-682e699d45fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:15:20.089725 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17cf987-21c4-4605-ae7a-682e699d45fb admin admin] PATH_INFO: `/v3/projects/ad5b351c4def40e0a94eeceec50cbffd/users/2e6f3cd755294dbca30f771764451da3/roles/3255f77192524a16bc405173106871fc` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:15:20.112993 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17cf987-21c4-4605-ae7a-682e699d45fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad5b351c4def40e0a94eeceec50cbffd, user_id=2e6f3cd755294dbca30f771764451da3, role_id=3255f77192524a16bc405173106871fc)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:15:20.114047 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: WARNING py.warnings [None req-c17cf987-21c4-4605-ae7a-682e699d45fb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 25 03:15:20.114047 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: warnings.warn(msg) Aug 25 03:15:20.114047 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: Aug 25 03:15:20.115999 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17cf987-21c4-4605-ae7a-682e699d45fb admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:15:20.223655 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 758/1501] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:15:19 2022] PUT /identity/v3/projects/ad5b351c4def40e0a94eeceec50cbffd/users/2e6f3cd755294dbca30f771764451da3/roles/3255f77192524a16bc405173106871fc => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:15:20.236398 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ce38d5-3c9a-4e65-bb97-ad7b01fe4db3 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:15:20.236683 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ce38d5-3c9a-4e65-bb97-ad7b01fe4db3 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:15:20.236955 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ce38d5-3c9a-4e65-bb97-ad7b01fe4db3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:15:20.292226 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-e8ce38d5-3c9a-4e65-bb97-ad7b01fe4db3 None None] MFA Rules not processed for user `2e6f3cd755294dbca30f771764451da3`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:15:20.293428 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-e8ce38d5-3c9a-4e65-bb97-ad7b01fe4db3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:15:20.367867 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 744/1502] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:15:20 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:15:20.381124 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd41feff-2bff-4e82-9ba2-aa62e9f9aea1 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:15:20.381409 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd41feff-2bff-4e82-9ba2-aa62e9f9aea1 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:15:20.381686 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd41feff-2bff-4e82-9ba2-aa62e9f9aea1 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:15:20.383122 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 759/1503] 10.222.0.58 () {54 vars in 964 bytes} [Thu Aug 25 03:15:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:15:20.387995 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b350a086-b1c9-42cb-9164-2211a13f79ec None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:15:20.388280 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b350a086-b1c9-42cb-9164-2211a13f79ec None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:15:20.388556 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b350a086-b1c9-42cb-9164-2211a13f79ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:15:20.420093 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:15:20.464360 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-b350a086-b1c9-42cb-9164-2211a13f79ec None None] MFA Rules not processed for user `60193ba9e1c14df3bc9eebcc6f28f2d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:15:20.465576 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-b350a086-b1c9-42cb-9164-2211a13f79ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:15:20.539152 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 745/1504] 10.222.0.58 () {58 vars in 1061 bytes} [Thu Aug 25 03:15:20 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:15:20.546767 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd41feff-2bff-4e82-9ba2-aa62e9f9aea1 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:15:20.548281 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-dd41feff-2bff-4e82-9ba2-aa62e9f9aea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:15:20.598973 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:15:20.653566 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd41feff-2bff-4e82-9ba2-aa62e9f9aea1 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:15:20.726712 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20c33d9-5ded-4442-8a72-4933dcc84bf9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60193ba9e1c14df3bc9eebcc6f28f2d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:15:20.727320 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20c33d9-5ded-4442-8a72-4933dcc84bf9 service cinder] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:15:20.727623 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20c33d9-5ded-4442-8a72-4933dcc84bf9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:15:20.727917 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20c33d9-5ded-4442-8a72-4933dcc84bf9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:15:20.729463 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-e20c33d9-5ded-4442-8a72-4933dcc84bf9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:15:20.825402 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20c33d9-5ded-4442-8a72-4933dcc84bf9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:15:20.826819 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20c33d9-5ded-4442-8a72-4933dcc84bf9 service cinder] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:15:20.830543 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:15:20.844715 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 760/1505] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:15:20 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:15:20.890511 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0144f2-34c0-4bcf-8cb3-32fafe30392b None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:15:20.890942 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0144f2-34c0-4bcf-8cb3-32fafe30392b None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:15:20.891064 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0144f2-34c0-4bcf-8cb3-32fafe30392b None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:15:20.892348 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 746/1506] 10.222.0.58 () {52 vars in 830 bytes} [Thu Aug 25 03:15:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:15:20.895963 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20c33d9-5ded-4442-8a72-4933dcc84bf9 service cinder] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:15:20.897206 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-e20c33d9-5ded-4442-8a72-4933dcc84bf9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:15:20.995734 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20c33d9-5ded-4442-8a72-4933dcc84bf9 service cinder] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:15:21.087965 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e4c7df-b16f-4c71-8c07-2461def140b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:15:21.088528 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e4c7df-b16f-4c71-8c07-2461def140b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:15:21.088803 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e4c7df-b16f-4c71-8c07-2461def140b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:15:21.089077 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e4c7df-b16f-4c71-8c07-2461def140b6 admin admin] PATH_INFO: `/v3/users/2e6f3cd755294dbca30f771764451da3` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:15:21.101191 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e4c7df-b16f-4c71-8c07-2461def140b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e6f3cd755294dbca30f771764451da3)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:15:21.102204 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: WARNING py.warnings [None req-90e4c7df-b16f-4c71-8c07-2461def140b6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 25 03:15:21.102204 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: warnings.warn(msg) Aug 25 03:15:21.102204 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: Aug 25 03:15:21.103766 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e4c7df-b16f-4c71-8c07-2461def140b6 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:15:21.114344 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:15:21.304865 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-90e4c7df-b16f-4c71-8c07-2461def140b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e6f3cd755294dbca30f771764451da3'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:15:21.381974 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-90e4c7df-b16f-4c71-8c07-2461def140b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e6f3cd755294dbca30f771764451da3'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:15:21.382482 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-90e4c7df-b16f-4c71-8c07-2461def140b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e6f3cd755294dbca30f771764451da3'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:15:21.396003 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-90e4c7df-b16f-4c71-8c07-2461def140b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e6f3cd755294dbca30f771764451da3'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:15:21.418599 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-90e4c7df-b16f-4c71-8c07-2461def140b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e6f3cd755294dbca30f771764451da3'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:15:21.424056 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 761/1507] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:15:20 2022] DELETE /identity/v3/users/2e6f3cd755294dbca30f771764451da3 => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:15:21.428492 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469305fe-0f94-4794-a1f3-4c417a3d01a6 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:15:21.432520 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469305fe-0f94-4794-a1f3-4c417a3d01a6 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:15:21.432520 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469305fe-0f94-4794-a1f3-4c417a3d01a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:15:21.491016 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-469305fe-0f94-4794-a1f3-4c417a3d01a6 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:15:21.492234 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-469305fe-0f94-4794-a1f3-4c417a3d01a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:15:21.566930 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:15:21.579308 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 747/1508] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:15:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:15:21.590481 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e4c7df-b16f-4c71-8c07-2461def140b6 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:15:21.591830 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-90e4c7df-b16f-4c71-8c07-2461def140b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:15:21.692872 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e4c7df-b16f-4c71-8c07-2461def140b6 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:15:21.747645 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:15:21.766142 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f34a452-2a36-48fb-9cb0-ad54cce48265 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:15:21.766811 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f34a452-2a36-48fb-9cb0-ad54cce48265 service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:15:21.767094 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f34a452-2a36-48fb-9cb0-ad54cce48265 service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:15:21.767371 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f34a452-2a36-48fb-9cb0-ad54cce48265 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:15:21.768971 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-3f34a452-2a36-48fb-9cb0-ad54cce48265 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:15:21.876946 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f34a452-2a36-48fb-9cb0-ad54cce48265 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:15:21.878470 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f34a452-2a36-48fb-9cb0-ad54cce48265 service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:15:21.896799 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 762/1509] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:15:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:15:22.337966 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469305fe-0f94-4794-a1f3-4c417a3d01a6 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:15:22.340118 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-469305fe-0f94-4794-a1f3-4c417a3d01a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:15:22.350904 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469305fe-0f94-4794-a1f3-4c417a3d01a6 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:15:22.448931 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad0d1ea-8911-47bb-8eaa-d95ea9694fdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:15:22.449563 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad0d1ea-8911-47bb-8eaa-d95ea9694fdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:15:22.449842 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad0d1ea-8911-47bb-8eaa-d95ea9694fdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:15:22.450113 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad0d1ea-8911-47bb-8eaa-d95ea9694fdf admin admin] PATH_INFO: `/v3/projects/ad5b351c4def40e0a94eeceec50cbffd` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:15:22.459789 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad0d1ea-8911-47bb-8eaa-d95ea9694fdf admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad5b351c4def40e0a94eeceec50cbffd)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:15:22.460812 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: WARNING py.warnings [None req-2ad0d1ea-8911-47bb-8eaa-d95ea9694fdf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 25 03:15:22.460812 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: warnings.warn(msg) Aug 25 03:15:22.460812 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: Aug 25 03:15:22.462029 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad0d1ea-8911-47bb-8eaa-d95ea9694fdf admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:15:22.569086 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-2ad0d1ea-8911-47bb-8eaa-d95ea9694fdf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad5b351c4def40e0a94eeceec50cbffd', 'request_id': 'req-2ad0d1ea-8911-47bb-8eaa-d95ea9694fdf'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:15:22.656125 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-2ad0d1ea-8911-47bb-8eaa-d95ea9694fdf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad5b351c4def40e0a94eeceec50cbffd', 'request_id': 'req-2ad0d1ea-8911-47bb-8eaa-d95ea9694fdf'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:15:22.662950 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-2ad0d1ea-8911-47bb-8eaa-d95ea9694fdf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad5b351c4def40e0a94eeceec50cbffd', 'request_id': 'req-2ad0d1ea-8911-47bb-8eaa-d95ea9694fdf'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:15:22.693031 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-2ad0d1ea-8911-47bb-8eaa-d95ea9694fdf admin admin] The token cache is being invalidate because project ad5b351c4def40e0a94eeceec50cbffd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81995) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:15:22.693325 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-2ad0d1ea-8911-47bb-8eaa-d95ea9694fdf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:15:22.693761 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-2ad0d1ea-8911-47bb-8eaa-d95ea9694fdf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:15:22.695224 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 748/1510] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:15:22 2022] DELETE /identity/v3/projects/ad5b351c4def40e0a94eeceec50cbffd => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:15:31.894633 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=62 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:15:32.690948 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:16:51.474209 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9890be4c-9a98-4855-aea7-a272a7d8d88e None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:16:51.474845 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9890be4c-9a98-4855-aea7-a272a7d8d88e None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:16:51.474845 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9890be4c-9a98-4855-aea7-a272a7d8d88e None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:16:51.476144 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 763/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 25 03:16:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:16:51.480944 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88bf910-3372-4dea-8572-95448e7e9b82 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:16:51.481366 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88bf910-3372-4dea-8572-95448e7e9b82 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:16:51.481499 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88bf910-3372-4dea-8572-95448e7e9b82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:16:51.564673 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-e88bf910-3372-4dea-8572-95448e7e9b82 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:16:51.566631 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-e88bf910-3372-4dea-8572-95448e7e9b82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:16:51.673464 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 749/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 25 03:16:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:16:51.829087 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9890be4c-9a98-4855-aea7-a272a7d8d88e None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:16:51.831240 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-9890be4c-9a98-4855-aea7-a272a7d8d88e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:16:51.949382 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9890be4c-9a98-4855-aea7-a272a7d8d88e None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:16:51.956163 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5adb143-22e7-4f40-a2c8-967a01da0fc4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cd171bf4df94f8fab397e42835feeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:16:51.956806 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5adb143-22e7-4f40-a2c8-967a01da0fc4 service nova] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:16:51.957087 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5adb143-22e7-4f40-a2c8-967a01da0fc4 service nova] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:16:51.957361 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5adb143-22e7-4f40-a2c8-967a01da0fc4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:16:51.966804 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5adb143-22e7-4f40-a2c8-967a01da0fc4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:16:51.968390 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5adb143-22e7-4f40-a2c8-967a01da0fc4 service nova] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:16:51.977523 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 764/1513] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:16:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:16:52.810041 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b60f68c-9ba3-4033-bd92-5524f851e2aa None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:16:52.810626 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b60f68c-9ba3-4033-bd92-5524f851e2aa None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:16:52.810626 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b60f68c-9ba3-4033-bd92-5524f851e2aa None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:16:52.811959 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 750/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 25 03:16:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:16:52.818167 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec889b4-974f-4dd0-b054-a759982614b3 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:16:52.818611 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec889b4-974f-4dd0-b054-a759982614b3 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:16:52.818753 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec889b4-974f-4dd0-b054-a759982614b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:16:52.838734 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-aec889b4-974f-4dd0-b054-a759982614b3 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:16:52.840063 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-aec889b4-974f-4dd0-b054-a759982614b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:16:52.847964 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 765/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 25 03:16:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:16:53.002690 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b60f68c-9ba3-4033-bd92-5524f851e2aa None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:16:53.017979 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b60f68c-9ba3-4033-bd92-5524f851e2aa None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:16:53.023367 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897cb4ad-6568-44fe-84bd-6dc4d4720577 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cd171bf4df94f8fab397e42835feeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:16:53.023963 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897cb4ad-6568-44fe-84bd-6dc4d4720577 service nova] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:16:53.024241 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897cb4ad-6568-44fe-84bd-6dc4d4720577 service nova] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:16:53.024512 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897cb4ad-6568-44fe-84bd-6dc4d4720577 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:16:53.032187 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c24c16-5647-4a00-a46a-8bb20f2d170a None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:16:53.032480 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c24c16-5647-4a00-a46a-8bb20f2d170a None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:16:53.032764 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c24c16-5647-4a00-a46a-8bb20f2d170a None None] PATH_INFO: `/v3` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:16:53.033062 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897cb4ad-6568-44fe-84bd-6dc4d4720577 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:16:53.034085 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 766/1516] 10.222.0.17 () {54 vars in 925 bytes} [Thu Aug 25 03:16:53 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:16:53.034674 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897cb4ad-6568-44fe-84bd-6dc4d4720577 service nova] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:16:53.039761 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6847cf3-7f30-4da1-8ddb-77c30e2a0564 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:16:53.040125 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6847cf3-7f30-4da1-8ddb-77c30e2a0564 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:16:53.040412 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6847cf3-7f30-4da1-8ddb-77c30e2a0564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:16:53.044864 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 751/1517] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:16:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:16:53.105328 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-f6847cf3-7f30-4da1-8ddb-77c30e2a0564 None None] MFA Rules not processed for user `2b692edd3e8c41289cc42e1afb4a6dd8`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:16:53.106646 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-f6847cf3-7f30-4da1-8ddb-77c30e2a0564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:16:53.161935 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:16:53.180246 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 767/1518] 10.222.0.17 () {58 vars in 1015 bytes} [Thu Aug 25 03:16:53 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:17:03.040021 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=75 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:17:03.193974 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=4 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:17:03.556562 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f461eec1-5363-41ef-8977-00b4060b57ca None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:03.556850 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f461eec1-5363-41ef-8977-00b4060b57ca None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:03.557131 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f461eec1-5363-41ef-8977-00b4060b57ca None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:03.558821 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 752/1519] 10.222.0.58 () {54 vars in 961 bytes} [Thu Aug 25 03:17:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:17:03.563888 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b638880b-0284-4d96-9c45-6987410b58fd None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:03.564259 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b638880b-0284-4d96-9c45-6987410b58fd None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:03.564536 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b638880b-0284-4d96-9c45-6987410b58fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:03.584555 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-b638880b-0284-4d96-9c45-6987410b58fd None None] MFA Rules not processed for user `2b692edd3e8c41289cc42e1afb4a6dd8`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:17:03.585758 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-b638880b-0284-4d96-9c45-6987410b58fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:17:03.595065 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 768/1520] 10.222.0.58 () {58 vars in 1058 bytes} [Thu Aug 25 03:17:03 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:17:03.602594 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f461eec1-5363-41ef-8977-00b4060b57ca None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:17:03.620313 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f461eec1-5363-41ef-8977-00b4060b57ca None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:17:03.625382 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71482ce6-b6ad-4ae9-a747-69a5af451755 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b692edd3e8c41289cc42e1afb4a6dd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:17:03.625990 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71482ce6-b6ad-4ae9-a747-69a5af451755 service placement] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:03.626272 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71482ce6-b6ad-4ae9-a747-69a5af451755 service placement] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:03.626580 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71482ce6-b6ad-4ae9-a747-69a5af451755 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:03.635749 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71482ce6-b6ad-4ae9-a747-69a5af451755 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:17:03.637906 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71482ce6-b6ad-4ae9-a747-69a5af451755 service placement] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:17:03.649534 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 753/1521] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:17:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:17:04.297924 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac20c8c5-6b22-40de-a251-39b85651e9a7 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:04.298455 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac20c8c5-6b22-40de-a251-39b85651e9a7 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:04.298597 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac20c8c5-6b22-40de-a251-39b85651e9a7 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:04.300048 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 769/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 25 03:17:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:17:04.304997 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7cdf98-72d7-4a45-83ac-344e368c20bb None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:04.305417 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7cdf98-72d7-4a45-83ac-344e368c20bb None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:04.305579 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7cdf98-72d7-4a45-83ac-344e368c20bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:04.325886 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-db7cdf98-72d7-4a45-83ac-344e368c20bb None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:17:04.327113 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-db7cdf98-72d7-4a45-83ac-344e368c20bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:17:04.336721 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 754/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 25 03:17:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:17:04.495946 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac20c8c5-6b22-40de-a251-39b85651e9a7 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:17:04.512032 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac20c8c5-6b22-40de-a251-39b85651e9a7 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:17:04.517666 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d79f7a-b0ea-477b-a352-ccfcf1dc325a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cd171bf4df94f8fab397e42835feeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:17:04.518261 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d79f7a-b0ea-477b-a352-ccfcf1dc325a service nova] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:04.518569 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d79f7a-b0ea-477b-a352-ccfcf1dc325a service nova] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:04.518840 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d79f7a-b0ea-477b-a352-ccfcf1dc325a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:04.527155 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d79f7a-b0ea-477b-a352-ccfcf1dc325a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:17:04.530006 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d79f7a-b0ea-477b-a352-ccfcf1dc325a service nova] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:17:04.538721 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 770/1524] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:17:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:17:05.362873 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2a3048-d52b-4976-8c3d-2be6d2b0af17 None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:05.363446 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2a3048-d52b-4976-8c3d-2be6d2b0af17 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:05.363446 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2a3048-d52b-4976-8c3d-2be6d2b0af17 None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:05.364770 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 755/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 25 03:17:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:17:05.369664 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee9231e-7540-4e9e-ac24-4853191ec90f None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:05.370101 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee9231e-7540-4e9e-ac24-4853191ec90f None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:05.370201 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee9231e-7540-4e9e-ac24-4853191ec90f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:05.389710 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-fee9231e-7540-4e9e-ac24-4853191ec90f None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:17:05.390923 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-fee9231e-7540-4e9e-ac24-4853191ec90f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:17:05.398727 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 771/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 25 03:17:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:17:05.551779 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2a3048-d52b-4976-8c3d-2be6d2b0af17 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:17:05.565885 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2a3048-d52b-4976-8c3d-2be6d2b0af17 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:17:05.571307 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5fb217-bcb4-4611-869d-f09b13ed8f0c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cd171bf4df94f8fab397e42835feeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:17:05.571898 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5fb217-bcb4-4611-869d-f09b13ed8f0c service nova] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:05.572178 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5fb217-bcb4-4611-869d-f09b13ed8f0c service nova] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:05.572699 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5fb217-bcb4-4611-869d-f09b13ed8f0c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:05.580740 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5fb217-bcb4-4611-869d-f09b13ed8f0c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:17:05.582351 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5fb217-bcb4-4611-869d-f09b13ed8f0c service nova] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:17:05.592254 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 756/1527] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:17:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:17:09.606774 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7073f3cb-b7bf-4f59-927e-e79ffd23b202 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:09.607350 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7073f3cb-b7bf-4f59-927e-e79ffd23b202 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:09.607350 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7073f3cb-b7bf-4f59-927e-e79ffd23b202 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:09.608663 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 772/1528] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 25 03:17:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:17:09.614219 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4157b4cf-30a6-4896-96bf-b2943ba6a726 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:09.614673 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4157b4cf-30a6-4896-96bf-b2943ba6a726 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:09.614778 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4157b4cf-30a6-4896-96bf-b2943ba6a726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:09.633550 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-4157b4cf-30a6-4896-96bf-b2943ba6a726 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:17:09.634782 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-4157b4cf-30a6-4896-96bf-b2943ba6a726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:17:09.643090 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 757/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 25 03:17:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:17:09.938048 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7073f3cb-b7bf-4f59-927e-e79ffd23b202 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:17:09.939788 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-7073f3cb-b7bf-4f59-927e-e79ffd23b202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:17:10.018924 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7073f3cb-b7bf-4f59-927e-e79ffd23b202 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:17:10.022483 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cf58b3-e311-446b-8418-f73dc65c72bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:17:10.023095 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf58b3-e311-446b-8418-f73dc65c72bd service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:10.023379 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf58b3-e311-446b-8418-f73dc65c72bd service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:10.023650 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf58b3-e311-446b-8418-f73dc65c72bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:10.032971 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cf58b3-e311-446b-8418-f73dc65c72bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:17:10.034429 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cf58b3-e311-446b-8418-f73dc65c72bd service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:17:10.044224 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 773/1530] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:17:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:17:11.947919 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc7081b-7778-421a-b59c-4cf3445dfdeb None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:11.948449 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc7081b-7778-421a-b59c-4cf3445dfdeb None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:11.948449 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc7081b-7778-421a-b59c-4cf3445dfdeb None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:11.950311 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 758/1531] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 25 03:17:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:17:11.955721 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a729de7-0da1-47cc-8272-c68c8eec493b None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:11.956133 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a729de7-0da1-47cc-8272-c68c8eec493b None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:11.956262 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a729de7-0da1-47cc-8272-c68c8eec493b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:11.975815 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-4a729de7-0da1-47cc-8272-c68c8eec493b None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:17:11.976987 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-4a729de7-0da1-47cc-8272-c68c8eec493b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:17:11.985014 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 774/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 25 03:17:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:17:12.274446 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc7081b-7778-421a-b59c-4cf3445dfdeb None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:17:12.285371 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc7081b-7778-421a-b59c-4cf3445dfdeb None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:17:12.288827 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcbf87e-075e-4b06-af37-bee5d2c068e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:17:12.289413 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcbf87e-075e-4b06-af37-bee5d2c068e6 service neutron] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:12.289689 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcbf87e-075e-4b06-af37-bee5d2c068e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:12.289957 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcbf87e-075e-4b06-af37-bee5d2c068e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:12.298647 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcbf87e-075e-4b06-af37-bee5d2c068e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:17:12.300051 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcbf87e-075e-4b06-af37-bee5d2c068e6 service neutron] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:17:12.309635 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 759/1533] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:17:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:17:15.809942 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8feea5-2a18-4f09-a456-276824ff9e15 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:15.810443 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8feea5-2a18-4f09-a456-276824ff9e15 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:15.810563 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8feea5-2a18-4f09-a456-276824ff9e15 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:15.811938 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 775/1534] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 25 03:17:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:17:15.817183 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaaf7f3-8857-4f5b-9473-fb8b955ac39e None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:15.817588 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaaf7f3-8857-4f5b-9473-fb8b955ac39e None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:15.817766 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaaf7f3-8857-4f5b-9473-fb8b955ac39e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:15.837370 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-aeaaf7f3-8857-4f5b-9473-fb8b955ac39e None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:17:15.838596 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-aeaaf7f3-8857-4f5b-9473-fb8b955ac39e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:17:15.847046 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 760/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 25 03:17:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:17:16.397859 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8feea5-2a18-4f09-a456-276824ff9e15 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:17:16.399620 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-7b8feea5-2a18-4f09-a456-276824ff9e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:17:16.477189 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8feea5-2a18-4f09-a456-276824ff9e15 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:17:16.480787 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e88c5a-c5a7-41df-a4e5-d399ca815e43 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60193ba9e1c14df3bc9eebcc6f28f2d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:17:16.481395 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e88c5a-c5a7-41df-a4e5-d399ca815e43 service cinder] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:16.481677 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e88c5a-c5a7-41df-a4e5-d399ca815e43 service cinder] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:16.481951 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e88c5a-c5a7-41df-a4e5-d399ca815e43 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:16.490660 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e88c5a-c5a7-41df-a4e5-d399ca815e43 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:17:16.492084 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e88c5a-c5a7-41df-a4e5-d399ca815e43 service cinder] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:17:16.501261 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 776/1536] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:17:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:17:18.460888 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468a5f13-b5e9-4724-857c-58231969c45c None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:18.461460 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468a5f13-b5e9-4724-857c-58231969c45c None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:18.461460 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468a5f13-b5e9-4724-857c-58231969c45c None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:18.462796 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 761/1537] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 25 03:17:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:17:18.468255 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd7789b-986c-4d1f-8546-3f90a62fe8cc None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:18.468707 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd7789b-986c-4d1f-8546-3f90a62fe8cc None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:18.468819 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd7789b-986c-4d1f-8546-3f90a62fe8cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:18.487628 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-4fd7789b-986c-4d1f-8546-3f90a62fe8cc None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:17:18.488817 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-4fd7789b-986c-4d1f-8546-3f90a62fe8cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:17:18.497057 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 777/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 25 03:17:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:17:19.055101 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468a5f13-b5e9-4724-857c-58231969c45c None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:17:19.056788 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-468a5f13-b5e9-4724-857c-58231969c45c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:17:19.069622 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468a5f13-b5e9-4724-857c-58231969c45c None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:17:19.073548 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637f07ab-1b75-49f8-bf19-ee669edc3473 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60193ba9e1c14df3bc9eebcc6f28f2d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:17:19.076159 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637f07ab-1b75-49f8-bf19-ee669edc3473 service cinder] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:19.076434 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637f07ab-1b75-49f8-bf19-ee669edc3473 service cinder] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:19.076699 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637f07ab-1b75-49f8-bf19-ee669edc3473 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:19.085727 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637f07ab-1b75-49f8-bf19-ee669edc3473 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:17:19.087255 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637f07ab-1b75-49f8-bf19-ee669edc3473 service cinder] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:17:19.097114 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 762/1539] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:17:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:17:28.483076 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=88 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:17:29.092952 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=33 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:18:44.195681 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef055760-fde9-4281-bd15-dfa133b62b69 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:44.196399 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef055760-fde9-4281-bd15-dfa133b62b69 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:44.196399 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef055760-fde9-4281-bd15-dfa133b62b69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:44.223322 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-ef055760-fde9-4281-bd15-dfa133b62b69 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:18:44.224711 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-ef055760-fde9-4281-bd15-dfa133b62b69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:18:44.236771 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 778/1540] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:18:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:18:44.245119 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74426d04-f0a3-4005-8d25-ee0867533a34 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:44.245705 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74426d04-f0a3-4005-8d25-ee0867533a34 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:44.245705 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74426d04-f0a3-4005-8d25-ee0867533a34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:44.281010 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-74426d04-f0a3-4005-8d25-ee0867533a34 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:18:44.282249 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-74426d04-f0a3-4005-8d25-ee0867533a34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:18:44.297463 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 763/1541] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:18:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:18:44.302811 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef055760-fde9-4281-bd15-dfa133b62b69 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:44.305159 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc731a05-b5e7-45c1-93c9-231ab0c0d391 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:44.305447 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc731a05-b5e7-45c1-93c9-231ab0c0d391 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:44.305726 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc731a05-b5e7-45c1-93c9-231ab0c0d391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:44.318712 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef055760-fde9-4281-bd15-dfa133b62b69 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:44.331991 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c981ac-4e87-421b-b5a1-796462bfa212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:44.332626 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c981ac-4e87-421b-b5a1-796462bfa212 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:44.332912 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c981ac-4e87-421b-b5a1-796462bfa212 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:44.333191 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c981ac-4e87-421b-b5a1-796462bfa212 admin admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:44.333849 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c981ac-4e87-421b-b5a1-796462bfa212 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:18:44.334153 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c981ac-4e87-421b-b5a1-796462bfa212 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:44.335016 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-bc731a05-b5e7-45c1-93c9-231ab0c0d391 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:18:44.335229 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: WARNING py.warnings [None req-36c981ac-4e87-421b-b5a1-796462bfa212 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 25 03:18:44.335229 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: warnings.warn(msg) Aug 25 03:18:44.335229 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: Aug 25 03:18:44.338999 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c981ac-4e87-421b-b5a1-796462bfa212 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:44.339107 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-bc731a05-b5e7-45c1-93c9-231ab0c0d391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:18:44.351219 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 764/1542] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:18:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:18:44.355161 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 779/1543] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Aug 25 03:18:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:18:44.396007 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc731a05-b5e7-45c1-93c9-231ab0c0d391 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:44.397570 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0030c290-8cc0-47cf-baeb-ddd80b0eb2ca None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:44.397570 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0030c290-8cc0-47cf-baeb-ddd80b0eb2ca None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:44.397570 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0030c290-8cc0-47cf-baeb-ddd80b0eb2ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:44.412889 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc731a05-b5e7-45c1-93c9-231ab0c0d391 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:44.425217 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-0030c290-8cc0-47cf-baeb-ddd80b0eb2ca None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:18:44.426467 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-0030c290-8cc0-47cf-baeb-ddd80b0eb2ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:18:44.428235 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f39957-693c-433b-89d8-e1b7ddd78f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:44.428903 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f39957-693c-433b-89d8-e1b7ddd78f06 admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:44.429205 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f39957-693c-433b-89d8-e1b7ddd78f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:44.429462 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f39957-693c-433b-89d8-e1b7ddd78f06 admin admin] PATH_INFO: `/v3/projects` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:44.430133 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f39957-693c-433b-89d8-e1b7ddd78f06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:44.431355 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: WARNING py.warnings [None req-27f39957-693c-433b-89d8-e1b7ddd78f06 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 25 03:18:44.431355 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: warnings.warn(msg) Aug 25 03:18:44.431355 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: Aug 25 03:18:44.432679 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f39957-693c-433b-89d8-e1b7ddd78f06 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:44.459425 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 780/1544] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:18:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:18:44.462853 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f204d99-5ae3-4397-aa10-655e6ead609c None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:44.463140 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f204d99-5ae3-4397-aa10-655e6ead609c None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:44.463488 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f204d99-5ae3-4397-aa10-655e6ead609c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:44.523021 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-2f204d99-5ae3-4397-aa10-655e6ead609c None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:18:44.524293 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-2f204d99-5ae3-4397-aa10-655e6ead609c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:18:44.533572 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 781/1545] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:18:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:18:44.534879 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de5121b-36da-4421-9bca-45118a5f6076 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:44.535171 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de5121b-36da-4421-9bca-45118a5f6076 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:44.535449 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de5121b-36da-4421-9bca-45118a5f6076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:44.567835 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-1de5121b-36da-4421-9bca-45118a5f6076 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:18:44.569163 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-1de5121b-36da-4421-9bca-45118a5f6076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:18:44.591317 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 765/1546] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:18:44 2022] POST /identity/v3/projects => generated 365 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:18:44.591317 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f39957-693c-433b-89d8-e1b7ddd78f06 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:44.594197 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 782/1547] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:18:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:18:44.594544 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fcbfbf-7c5a-4894-a3bd-4eb509f97e5b None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:44.594829 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fcbfbf-7c5a-4894-a3bd-4eb509f97e5b None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:44.595109 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fcbfbf-7c5a-4894-a3bd-4eb509f97e5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:44.622841 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-23fcbfbf-7c5a-4894-a3bd-4eb509f97e5b None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:18:44.624652 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-23fcbfbf-7c5a-4894-a3bd-4eb509f97e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:18:44.671430 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f39957-693c-433b-89d8-e1b7ddd78f06 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:44.676357 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9ccf29-0282-43f1-bcdd-ad2e0b578659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:44.676681 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9ccf29-0282-43f1-bcdd-ad2e0b578659 admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:44.676958 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9ccf29-0282-43f1-bcdd-ad2e0b578659 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:44.677233 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9ccf29-0282-43f1-bcdd-ad2e0b578659 admin admin] PATH_INFO: `/v3/domains` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:44.677876 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9ccf29-0282-43f1-bcdd-ad2e0b578659 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:18:44.678179 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9ccf29-0282-43f1-bcdd-ad2e0b578659 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:44.679820 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9ccf29-0282-43f1-bcdd-ad2e0b578659 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:44.691557 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 766/1548] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Aug 25 03:18:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:18:44.692884 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5f488e-8263-4d71-81ba-512b1a1f401f None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:44.693176 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5f488e-8263-4d71-81ba-512b1a1f401f None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:44.693453 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5f488e-8263-4d71-81ba-512b1a1f401f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:44.700827 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 783/1549] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:18:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:18:44.700827 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fcbfbf-7c5a-4894-a3bd-4eb509f97e5b None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:44.708156 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fcbfbf-7c5a-4894-a3bd-4eb509f97e5b None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:44.712822 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc6a176-ee22-430d-9484-8b0215c5121e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:44.713423 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc6a176-ee22-430d-9484-8b0215c5121e admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:44.713708 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc6a176-ee22-430d-9484-8b0215c5121e admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:44.713990 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc6a176-ee22-430d-9484-8b0215c5121e admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:44.714654 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc6a176-ee22-430d-9484-8b0215c5121e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:44.715694 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: WARNING py.warnings [None req-1cc6a176-ee22-430d-9484-8b0215c5121e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 25 03:18:44.715694 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: warnings.warn(msg) Aug 25 03:18:44.715694 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: Aug 25 03:18:44.716678 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-6b5f488e-8263-4d71-81ba-512b1a1f401f None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:18:44.716899 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc6a176-ee22-430d-9484-8b0215c5121e admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:44.717880 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-6b5f488e-8263-4d71-81ba-512b1a1f401f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:18:44.725666 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 767/1550] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:18:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:18:44.727040 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca7ec64-3316-4886-a083-b5d4fc0218aa None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:44.727334 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca7ec64-3316-4886-a083-b5d4fc0218aa None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:44.727611 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca7ec64-3316-4886-a083-b5d4fc0218aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:44.746468 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-7ca7ec64-3316-4886-a083-b5d4fc0218aa None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:18:44.748182 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-7ca7ec64-3316-4886-a083-b5d4fc0218aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:18:44.756046 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 768/1551] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:18:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:18:44.757316 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39f4286-ab42-4d1e-88d9-b21c7ffd42e6 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:44.757601 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39f4286-ab42-4d1e-88d9-b21c7ffd42e6 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:44.757873 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39f4286-ab42-4d1e-88d9-b21c7ffd42e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:44.777650 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-a39f4286-ab42-4d1e-88d9-b21c7ffd42e6 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:18:44.778878 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-a39f4286-ab42-4d1e-88d9-b21c7ffd42e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:18:44.786761 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 769/1552] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:18:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:18:44.787897 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a900b271-cd55-4d05-bb88-8bdcbb08e55c None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:44.788177 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a900b271-cd55-4d05-bb88-8bdcbb08e55c None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:44.788450 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a900b271-cd55-4d05-bb88-8bdcbb08e55c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:44.808050 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-a900b271-cd55-4d05-bb88-8bdcbb08e55c None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:18:44.810127 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-a900b271-cd55-4d05-bb88-8bdcbb08e55c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:18:44.819734 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 770/1553] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:18:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:18:44.821065 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837ac01a-b8e2-47f5-abe2-dd8051a3a88a None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:44.821628 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837ac01a-b8e2-47f5-abe2-dd8051a3a88a None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:44.821904 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837ac01a-b8e2-47f5-abe2-dd8051a3a88a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:44.829129 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 784/1554] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:18:44 2022] POST /identity/v3/users => generated 283 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:18:44.831226 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc6a176-ee22-430d-9484-8b0215c5121e admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:44.843306 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc6a176-ee22-430d-9484-8b0215c5121e admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:44.845210 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-837ac01a-b8e2-47f5-abe2-dd8051a3a88a None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:18:44.846432 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-837ac01a-b8e2-47f5-abe2-dd8051a3a88a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:18:44.849745 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f0ad8a-c988-41a6-b008-0bd53ea675fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:44.850690 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f0ad8a-c988-41a6-b008-0bd53ea675fd admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:44.851241 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f0ad8a-c988-41a6-b008-0bd53ea675fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:44.851608 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f0ad8a-c988-41a6-b008-0bd53ea675fd admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:44.852416 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f0ad8a-c988-41a6-b008-0bd53ea675fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:44.854426 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 771/1555] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:18:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:18:44.855196 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837ac01a-b8e2-47f5-abe2-dd8051a3a88a None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:44.855616 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f0ad8a-c988-41a6-b008-0bd53ea675fd admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:44.866126 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837ac01a-b8e2-47f5-abe2-dd8051a3a88a None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:44.871013 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac84a14-3983-4ea9-bf36-ee956666b6da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:44.871800 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac84a14-3983-4ea9-bf36-ee956666b6da admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:44.872763 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac84a14-3983-4ea9-bf36-ee956666b6da admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:44.873042 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac84a14-3983-4ea9-bf36-ee956666b6da admin admin] PATH_INFO: `/v3/domains` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:44.873840 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac84a14-3983-4ea9-bf36-ee956666b6da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:18:44.874142 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac84a14-3983-4ea9-bf36-ee956666b6da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:44.875785 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac84a14-3983-4ea9-bf36-ee956666b6da admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:44.887614 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 772/1556] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Aug 25 03:18:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:18:44.888859 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abb552e-2d26-4687-83b5-2e910b411725 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:44.889144 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abb552e-2d26-4687-83b5-2e910b411725 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:44.889418 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abb552e-2d26-4687-83b5-2e910b411725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:44.909122 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-8abb552e-2d26-4687-83b5-2e910b411725 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:18:44.910335 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-8abb552e-2d26-4687-83b5-2e910b411725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:18:44.918225 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 773/1557] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:18:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:18:44.919566 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abb552e-2d26-4687-83b5-2e910b411725 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:44.928490 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abb552e-2d26-4687-83b5-2e910b411725 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:44.933061 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c19689-2bf5-4ff7-af45-7edb0bce8f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:44.933634 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c19689-2bf5-4ff7-af45-7edb0bce8f18 admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:44.933941 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c19689-2bf5-4ff7-af45-7edb0bce8f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:44.934241 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c19689-2bf5-4ff7-af45-7edb0bce8f18 admin admin] PATH_INFO: `/v3/domains` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:44.935455 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c19689-2bf5-4ff7-af45-7edb0bce8f18 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:18:44.935758 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c19689-2bf5-4ff7-af45-7edb0bce8f18 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:44.937334 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c19689-2bf5-4ff7-af45-7edb0bce8f18 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:44.949034 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 774/1558] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Aug 25 03:18:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:18:44.953190 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3401e5fb-6f46-493e-ae7f-0d7c2f1c855a None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:44.953190 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3401e5fb-6f46-493e-ae7f-0d7c2f1c855a None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:44.953190 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3401e5fb-6f46-493e-ae7f-0d7c2f1c855a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:44.969503 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-3401e5fb-6f46-493e-ae7f-0d7c2f1c855a None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:18:44.970896 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-3401e5fb-6f46-493e-ae7f-0d7c2f1c855a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:18:44.978808 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 775/1559] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:18:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:18:44.983959 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 785/1560] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:18:44 2022] POST /identity/v3/projects => generated 379 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:18:44.983959 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f0ad8a-c988-41a6-b008-0bd53ea675fd admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:44.984189 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8661c5d8-de7e-46a8-ba1a-d9e07310b44b None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:44.984189 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8661c5d8-de7e-46a8-ba1a-d9e07310b44b None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:44.984189 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8661c5d8-de7e-46a8-ba1a-d9e07310b44b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:45.002007 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-8661c5d8-de7e-46a8-ba1a-d9e07310b44b None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:18:45.003243 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-8661c5d8-de7e-46a8-ba1a-d9e07310b44b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:18:45.035874 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:18:45.045638 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f0ad8a-c988-41a6-b008-0bd53ea675fd admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:45.050321 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a16a271-c614-4d82-9f9d-0a3eec9e13e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:45.050937 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a16a271-c614-4d82-9f9d-0a3eec9e13e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:45.051227 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a16a271-c614-4d82-9f9d-0a3eec9e13e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:45.051502 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a16a271-c614-4d82-9f9d-0a3eec9e13e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:45.052147 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a16a271-c614-4d82-9f9d-0a3eec9e13e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:45.053739 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a16a271-c614-4d82-9f9d-0a3eec9e13e3 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:45.064613 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 786/1561] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:18:44 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:18:45.065853 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df035c49-5d14-49ba-94e3-24536aca7d30 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:45.066137 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df035c49-5d14-49ba-94e3-24536aca7d30 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:45.066447 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df035c49-5d14-49ba-94e3-24536aca7d30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:45.086221 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-df035c49-5d14-49ba-94e3-24536aca7d30 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:18:45.087676 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-df035c49-5d14-49ba-94e3-24536aca7d30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:18:45.093855 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 776/1562] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:18:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:18:45.097110 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 787/1563] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:18:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:18:45.098113 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df035c49-5d14-49ba-94e3-24536aca7d30 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:45.098459 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8661c5d8-de7e-46a8-ba1a-d9e07310b44b None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:45.108982 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8661c5d8-de7e-46a8-ba1a-d9e07310b44b None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:45.109149 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df035c49-5d14-49ba-94e3-24536aca7d30 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:45.113544 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4b1004-64b0-4f38-be33-30036320d3e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:45.113716 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee66093-8489-4d24-a5a3-58715f496c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:45.114131 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4b1004-64b0-4f38-be33-30036320d3e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:45.114343 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee66093-8489-4d24-a5a3-58715f496c83 admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:45.114461 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4b1004-64b0-4f38-be33-30036320d3e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:45.114634 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee66093-8489-4d24-a5a3-58715f496c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:45.114742 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4b1004-64b0-4f38-be33-30036320d3e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:45.114912 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee66093-8489-4d24-a5a3-58715f496c83 admin admin] PATH_INFO: `/v3/projects` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:45.115319 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4b1004-64b0-4f38-be33-30036320d3e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:18:45.115539 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee66093-8489-4d24-a5a3-58715f496c83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:45.115649 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4b1004-64b0-4f38-be33-30036320d3e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:45.117177 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4b1004-64b0-4f38-be33-30036320d3e8 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:45.117616 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee66093-8489-4d24-a5a3-58715f496c83 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:45.124723 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:18:45.130094 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 788/1564] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Aug 25 03:18:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:18:45.130881 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4b1004-64b0-4f38-be33-30036320d3e8 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:45.140737 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4b1004-64b0-4f38-be33-30036320d3e8 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:45.146105 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3ae61d-3789-4100-ab66-62931e00e3a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:45.146750 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3ae61d-3789-4100-ab66-62931e00e3a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:45.148853 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3ae61d-3789-4100-ab66-62931e00e3a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:45.149247 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3ae61d-3789-4100-ab66-62931e00e3a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:45.149886 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3ae61d-3789-4100-ab66-62931e00e3a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:45.151902 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3ae61d-3789-4100-ab66-62931e00e3a6 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:45.264791 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 777/1565] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:18:45 2022] POST /identity/v3/projects => generated 383 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:18:45.265611 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee66093-8489-4d24-a5a3-58715f496c83 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:45.276241 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 789/1566] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:18:45 2022] POST /identity/v3/projects => generated 375 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:18:45.277126 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3ae61d-3789-4100-ab66-62931e00e3a6 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:45.332450 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee66093-8489-4d24-a5a3-58715f496c83 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:45.337084 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef124b8-4c5e-492a-a64a-c29b2ecbfe47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:45.337685 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef124b8-4c5e-492a-a64a-c29b2ecbfe47 admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:45.337974 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef124b8-4c5e-492a-a64a-c29b2ecbfe47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:45.338251 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef124b8-4c5e-492a-a64a-c29b2ecbfe47 admin admin] PATH_INFO: `/v3/users` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:45.338891 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef124b8-4c5e-492a-a64a-c29b2ecbfe47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:45.339008 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3ae61d-3789-4100-ab66-62931e00e3a6 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:45.341375 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef124b8-4c5e-492a-a64a-c29b2ecbfe47 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:45.344127 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067d361a-3ac9-44aa-9db8-885e534b47d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:45.344729 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067d361a-3ac9-44aa-9db8-885e534b47d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:45.345012 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067d361a-3ac9-44aa-9db8-885e534b47d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:45.345344 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067d361a-3ac9-44aa-9db8-885e534b47d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:45.345986 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067d361a-3ac9-44aa-9db8-885e534b47d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:18:45.346303 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067d361a-3ac9-44aa-9db8-885e534b47d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:45.347897 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067d361a-3ac9-44aa-9db8-885e534b47d6 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:45.359976 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 790/1567] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Aug 25 03:18:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:18:45.360803 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067d361a-3ac9-44aa-9db8-885e534b47d6 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:45.372049 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067d361a-3ac9-44aa-9db8-885e534b47d6 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:45.376858 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad10f57e-f557-4a3e-a51f-587163dda87d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:45.377445 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad10f57e-f557-4a3e-a51f-587163dda87d admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:45.377722 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad10f57e-f557-4a3e-a51f-587163dda87d admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:45.377992 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad10f57e-f557-4a3e-a51f-587163dda87d admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:45.378661 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad10f57e-f557-4a3e-a51f-587163dda87d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:45.380237 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad10f57e-f557-4a3e-a51f-587163dda87d admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:45.390523 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 791/1568] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:18:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:18:45.391411 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad10f57e-f557-4a3e-a51f-587163dda87d admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:45.400641 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad10f57e-f557-4a3e-a51f-587163dda87d admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:45.406401 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662c935e-f883-4da4-b3c1-fbe2efcb59d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:45.406401 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662c935e-f883-4da4-b3c1-fbe2efcb59d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:45.406697 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662c935e-f883-4da4-b3c1-fbe2efcb59d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:45.406820 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662c935e-f883-4da4-b3c1-fbe2efcb59d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:45.407499 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662c935e-f883-4da4-b3c1-fbe2efcb59d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:18:45.407801 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662c935e-f883-4da4-b3c1-fbe2efcb59d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:45.409318 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662c935e-f883-4da4-b3c1-fbe2efcb59d7 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:45.420932 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 792/1569] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Aug 25 03:18:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:18:45.421803 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662c935e-f883-4da4-b3c1-fbe2efcb59d7 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:45.429935 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 778/1570] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:18:45 2022] POST /identity/v3/users => generated 290 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:18:45.431031 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef124b8-4c5e-492a-a64a-c29b2ecbfe47 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:45.432756 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662c935e-f883-4da4-b3c1-fbe2efcb59d7 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:45.437778 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dbc283-289d-471b-9665-d1581e4fb47a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:45.438392 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dbc283-289d-471b-9665-d1581e4fb47a admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:45.438885 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dbc283-289d-471b-9665-d1581e4fb47a admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:45.439154 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dbc283-289d-471b-9665-d1581e4fb47a admin admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:45.439784 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dbc283-289d-471b-9665-d1581e4fb47a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:18:45.440086 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dbc283-289d-471b-9665-d1581e4fb47a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:45.440803 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef124b8-4c5e-492a-a64a-c29b2ecbfe47 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:45.441656 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dbc283-289d-471b-9665-d1581e4fb47a admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:45.445411 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85734284-f815-4837-adc2-e6426c8fe0eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:45.446003 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85734284-f815-4837-adc2-e6426c8fe0eb admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:45.446295 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85734284-f815-4837-adc2-e6426c8fe0eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:45.446579 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85734284-f815-4837-adc2-e6426c8fe0eb admin admin] PATH_INFO: `/v3/projects` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:45.447195 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85734284-f815-4837-adc2-e6426c8fe0eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:45.449326 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85734284-f815-4837-adc2-e6426c8fe0eb admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:45.461531 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 793/1571] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Aug 25 03:18:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:18:45.461531 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dbc283-289d-471b-9665-d1581e4fb47a admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:45.467750 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dbc283-289d-471b-9665-d1581e4fb47a admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:45.472985 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe3d815-054e-4587-8ad1-b73c49380cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:45.477905 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe3d815-054e-4587-8ad1-b73c49380cc7 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:45.477905 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe3d815-054e-4587-8ad1-b73c49380cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:45.478148 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe3d815-054e-4587-8ad1-b73c49380cc7 admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:45.478682 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe3d815-054e-4587-8ad1-b73c49380cc7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:45.480642 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe3d815-054e-4587-8ad1-b73c49380cc7 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:45.535453 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 779/1572] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:18:45 2022] POST /identity/v3/projects => generated 377 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:18:45.536242 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85734284-f815-4837-adc2-e6426c8fe0eb admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:45.549514 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 794/1573] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:18:45 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:18:45.550199 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe3d815-054e-4587-8ad1-b73c49380cc7 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:45.559742 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe3d815-054e-4587-8ad1-b73c49380cc7 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:45.603892 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85734284-f815-4837-adc2-e6426c8fe0eb admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:45.609322 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61c0888-b6ab-4a85-8ef5-99b7254a972c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:45.609963 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61c0888-b6ab-4a85-8ef5-99b7254a972c admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:45.610245 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61c0888-b6ab-4a85-8ef5-99b7254a972c admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:45.610565 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61c0888-b6ab-4a85-8ef5-99b7254a972c admin admin] PATH_INFO: `/v3/users` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:45.611178 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61c0888-b6ab-4a85-8ef5-99b7254a972c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:45.613112 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61c0888-b6ab-4a85-8ef5-99b7254a972c admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:45.620894 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bec4628-d6e9-4e73-9fe3-64914b875af5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:45.621511 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bec4628-d6e9-4e73-9fe3-64914b875af5 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:45.621819 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bec4628-d6e9-4e73-9fe3-64914b875af5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:45.622205 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bec4628-d6e9-4e73-9fe3-64914b875af5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:45.622896 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bec4628-d6e9-4e73-9fe3-64914b875af5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:45.625029 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bec4628-d6e9-4e73-9fe3-64914b875af5 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:45.720334 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 780/1574] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:18:45 2022] POST /identity/v3/users => generated 288 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:18:45.721163 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61c0888-b6ab-4a85-8ef5-99b7254a972c admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:45.730695 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61c0888-b6ab-4a85-8ef5-99b7254a972c admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:45.735367 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f142e2b-c184-42a0-849a-5921a3925424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:45.735991 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f142e2b-c184-42a0-849a-5921a3925424 admin admin] REQUEST_METHOD: `PUT` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:45.736270 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f142e2b-c184-42a0-849a-5921a3925424 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:45.736543 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f142e2b-c184-42a0-849a-5921a3925424 admin admin] PATH_INFO: `/v3/projects/ffe88d801c9643c2a6cf9fce35080ad1/users/25be51a68ccf4ebbbaac8b4fd0633bc3/roles/3255f77192524a16bc405173106871fc` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:45.744463 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:18:45.747531 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:18:45.751148 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 795/1575] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:18:45 2022] POST /identity/v3/projects => generated 365 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:18:45.751980 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bec4628-d6e9-4e73-9fe3-64914b875af5 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:45.752165 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f142e2b-c184-42a0-849a-5921a3925424 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffe88d801c9643c2a6cf9fce35080ad1, user_id=25be51a68ccf4ebbbaac8b4fd0633bc3, role_id=3255f77192524a16bc405173106871fc)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:45.753196 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: WARNING py.warnings [None req-2f142e2b-c184-42a0-849a-5921a3925424 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 25 03:18:45.753196 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: warnings.warn(msg) Aug 25 03:18:45.753196 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: Aug 25 03:18:45.755424 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f142e2b-c184-42a0-849a-5921a3925424 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:45.814802 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bec4628-d6e9-4e73-9fe3-64914b875af5 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:45.819431 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cd8501-625b-4a5a-988b-64e4041c60cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:45.820045 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cd8501-625b-4a5a-988b-64e4041c60cb admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:45.820372 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cd8501-625b-4a5a-988b-64e4041c60cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:45.820592 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cd8501-625b-4a5a-988b-64e4041c60cb admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:45.821300 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cd8501-625b-4a5a-988b-64e4041c60cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:45.823590 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cd8501-625b-4a5a-988b-64e4041c60cb admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:45.863646 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 781/1576] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:18:45 2022] PUT /identity/v3/projects/ffe88d801c9643c2a6cf9fce35080ad1/users/25be51a68ccf4ebbbaac8b4fd0633bc3/roles/3255f77192524a16bc405173106871fc => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:18:45.864481 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f142e2b-c184-42a0-849a-5921a3925424 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:45.924945 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f142e2b-c184-42a0-849a-5921a3925424 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:45.929657 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cfbfbd-95a3-493e-ac97-bb433524b590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:45.930294 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cfbfbd-95a3-493e-ac97-bb433524b590 admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:45.930584 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cfbfbd-95a3-493e-ac97-bb433524b590 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:45.930860 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cfbfbd-95a3-493e-ac97-bb433524b590 admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:45.931505 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cfbfbd-95a3-493e-ac97-bb433524b590 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:45.933125 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cfbfbd-95a3-493e-ac97-bb433524b590 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:45.943209 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 782/1577] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:18:45 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:18:45.944007 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cfbfbd-95a3-493e-ac97-bb433524b590 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:45.953949 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cfbfbd-95a3-493e-ac97-bb433524b590 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:45.956787 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 796/1578] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:18:45 2022] POST /identity/v3/projects => generated 367 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:18:45.957755 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cd8501-625b-4a5a-988b-64e4041c60cb admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:45.958918 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4c9f34-a3af-432c-8bb2-b338b28c440c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:45.959505 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4c9f34-a3af-432c-8bb2-b338b28c440c admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:45.964015 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4c9f34-a3af-432c-8bb2-b338b28c440c admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:45.964015 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4c9f34-a3af-432c-8bb2-b338b28c440c admin admin] PATH_INFO: `/v3/projects` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:45.964015 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4c9f34-a3af-432c-8bb2-b338b28c440c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:45.964015 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4c9f34-a3af-432c-8bb2-b338b28c440c admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:46.018884 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cd8501-625b-4a5a-988b-64e4041c60cb admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:46.023426 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef0edf5-c171-487a-89fe-aec583518cf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:46.024036 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef0edf5-c171-487a-89fe-aec583518cf0 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:46.024313 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef0edf5-c171-487a-89fe-aec583518cf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:46.024582 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef0edf5-c171-487a-89fe-aec583518cf0 admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:46.025195 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef0edf5-c171-487a-89fe-aec583518cf0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:46.027159 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef0edf5-c171-487a-89fe-aec583518cf0 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:46.035775 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 783/1579] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:18:45 2022] POST /identity/v3/projects => generated 391 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:18:46.036804 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4c9f34-a3af-432c-8bb2-b338b28c440c admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:46.097380 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4c9f34-a3af-432c-8bb2-b338b28c440c admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:46.101909 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20c558c-531f-4247-893a-3008c74f44eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:46.102609 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20c558c-531f-4247-893a-3008c74f44eb admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:46.102894 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20c558c-531f-4247-893a-3008c74f44eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:46.103171 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20c558c-531f-4247-893a-3008c74f44eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:46.103811 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20c558c-531f-4247-893a-3008c74f44eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:46.105421 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20c558c-531f-4247-893a-3008c74f44eb admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:46.115072 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 784/1580] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:18:46 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:18:46.115877 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20c558c-531f-4247-893a-3008c74f44eb admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:46.126312 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 797/1581] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:18:45 2022] POST /identity/v3/users => generated 289 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:18:46.126487 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20c558c-531f-4247-893a-3008c74f44eb admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:46.127165 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef0edf5-c171-487a-89fe-aec583518cf0 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:46.132989 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7e9cf5-d99c-4844-ad45-aa23bedbf2cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:46.134659 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7e9cf5-d99c-4844-ad45-aa23bedbf2cf admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:46.134659 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7e9cf5-d99c-4844-ad45-aa23bedbf2cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:46.134659 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7e9cf5-d99c-4844-ad45-aa23bedbf2cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:46.135689 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7e9cf5-d99c-4844-ad45-aa23bedbf2cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:46.137331 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7e9cf5-d99c-4844-ad45-aa23bedbf2cf admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:46.138898 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef0edf5-c171-487a-89fe-aec583518cf0 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:46.143486 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6d827a-2ec7-42c1-83c9-0887ed55534c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:46.144084 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6d827a-2ec7-42c1-83c9-0887ed55534c admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:46.144363 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6d827a-2ec7-42c1-83c9-0887ed55534c admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:46.144636 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6d827a-2ec7-42c1-83c9-0887ed55534c admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:46.145253 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6d827a-2ec7-42c1-83c9-0887ed55534c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:46.147288 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6d827a-2ec7-42c1-83c9-0887ed55534c admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:46.148415 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 785/1582] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:18:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:18:46.150066 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe94b0a0-eacd-4c52-a8f7-bd61bb111a5a None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:46.150376 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe94b0a0-eacd-4c52-a8f7-bd61bb111a5a None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:46.150659 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe94b0a0-eacd-4c52-a8f7-bd61bb111a5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:46.178565 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-fe94b0a0-eacd-4c52-a8f7-bd61bb111a5a None None] MFA Rules not processed for user `25be51a68ccf4ebbbaac8b4fd0633bc3`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:18:46.179772 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-fe94b0a0-eacd-4c52-a8f7-bd61bb111a5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:18:46.218591 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:18:46.234592 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 798/1583] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:18:46 2022] POST /identity/v3/users => generated 283 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:18:46.237856 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6d827a-2ec7-42c1-83c9-0887ed55534c admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:46.244328 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 786/1584] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:18:46 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:18:46.245208 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe94b0a0-eacd-4c52-a8f7-bd61bb111a5a None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:46.246398 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6d827a-2ec7-42c1-83c9-0887ed55534c admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:46.252016 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fa2050-5d9e-4e30-9014-866576f91f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:46.252705 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fa2050-5d9e-4e30-9014-866576f91f09 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:46.253063 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fa2050-5d9e-4e30-9014-866576f91f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:46.253413 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fa2050-5d9e-4e30-9014-866576f91f09 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:46.254134 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fa2050-5d9e-4e30-9014-866576f91f09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:46.255898 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fa2050-5d9e-4e30-9014-866576f91f09 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:46.256163 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe94b0a0-eacd-4c52-a8f7-bd61bb111a5a None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:46.260868 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b051f31-0718-4447-9bcb-013ba75f8c0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:46.261499 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b051f31-0718-4447-9bcb-013ba75f8c0b admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:46.261777 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b051f31-0718-4447-9bcb-013ba75f8c0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:46.262049 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b051f31-0718-4447-9bcb-013ba75f8c0b admin admin] PATH_INFO: `/v3/users` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:46.263338 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b051f31-0718-4447-9bcb-013ba75f8c0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:46.265278 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b051f31-0718-4447-9bcb-013ba75f8c0b admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:46.269023 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 799/1585] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:18:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:18:46.269023 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fa2050-5d9e-4e30-9014-866576f91f09 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:46.279431 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fa2050-5d9e-4e30-9014-866576f91f09 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:46.284117 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ecba51-5951-4f13-8f4e-6f72cae255a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:46.284818 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ecba51-5951-4f13-8f4e-6f72cae255a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:46.285097 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ecba51-5951-4f13-8f4e-6f72cae255a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:46.285368 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ecba51-5951-4f13-8f4e-6f72cae255a8 admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:46.285986 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ecba51-5951-4f13-8f4e-6f72cae255a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:46.287924 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ecba51-5951-4f13-8f4e-6f72cae255a8 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:46.326385 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 787/1586] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:18:46 2022] POST /identity/v3/users => generated 284 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:18:46.327319 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b051f31-0718-4447-9bcb-013ba75f8c0b admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:46.336928 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b051f31-0718-4447-9bcb-013ba75f8c0b admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:46.341730 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7813bf-219a-453d-bbfa-404d9fc7f744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:46.342434 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7813bf-219a-453d-bbfa-404d9fc7f744 admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:46.342704 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7813bf-219a-453d-bbfa-404d9fc7f744 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:46.342977 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7813bf-219a-453d-bbfa-404d9fc7f744 admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:46.343668 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7813bf-219a-453d-bbfa-404d9fc7f744 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:46.345581 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7813bf-219a-453d-bbfa-404d9fc7f744 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:46.360699 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 800/1587] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:18:46 2022] POST /identity/v3/users => generated 296 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:18:46.360699 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ecba51-5951-4f13-8f4e-6f72cae255a8 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:46.360983 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 788/1588] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:18:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:18:46.360983 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7813bf-219a-453d-bbfa-404d9fc7f744 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:46.367397 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ecba51-5951-4f13-8f4e-6f72cae255a8 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:46.367567 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7813bf-219a-453d-bbfa-404d9fc7f744 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:46.372719 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b01de73-025e-491f-b9a1-a9370553828b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:46.372968 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d939e66-2c96-4158-94a0-626f447201a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:46.373318 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b01de73-025e-491f-b9a1-a9370553828b admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:46.373597 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b01de73-025e-491f-b9a1-a9370553828b admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:46.373846 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d939e66-2c96-4158-94a0-626f447201a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:46.373968 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b01de73-025e-491f-b9a1-a9370553828b admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:46.374247 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d939e66-2c96-4158-94a0-626f447201a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:46.374521 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b01de73-025e-491f-b9a1-a9370553828b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:46.374660 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d939e66-2c96-4158-94a0-626f447201a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:46.375544 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d939e66-2c96-4158-94a0-626f447201a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:46.376189 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b01de73-025e-491f-b9a1-a9370553828b admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:46.377401 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d939e66-2c96-4158-94a0-626f447201a8 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:46.386385 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 789/1589] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:18:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:18:46.387217 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b01de73-025e-491f-b9a1-a9370553828b admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:46.388052 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 801/1590] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:18:46 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:18:46.388893 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d939e66-2c96-4158-94a0-626f447201a8 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:46.399636 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d939e66-2c96-4158-94a0-626f447201a8 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:46.399808 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b01de73-025e-491f-b9a1-a9370553828b admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:46.404228 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8749a4a3-acbd-49f6-ac12-47f88cc52a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:46.404379 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f40859b-511a-46a5-98f2-d252b6164ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:46.404818 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8749a4a3-acbd-49f6-ac12-47f88cc52a8e admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:46.404923 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f40859b-511a-46a5-98f2-d252b6164ff6 admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:46.405090 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f40859b-511a-46a5-98f2-d252b6164ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:46.405196 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8749a4a3-acbd-49f6-ac12-47f88cc52a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:46.405374 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8749a4a3-acbd-49f6-ac12-47f88cc52a8e admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:46.405482 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f40859b-511a-46a5-98f2-d252b6164ff6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:46.405970 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8749a4a3-acbd-49f6-ac12-47f88cc52a8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:46.406085 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f40859b-511a-46a5-98f2-d252b6164ff6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:46.407727 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f40859b-511a-46a5-98f2-d252b6164ff6 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:46.411635 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8749a4a3-acbd-49f6-ac12-47f88cc52a8e admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:46.419865 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 790/1591] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:18:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:18:46.420743 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f40859b-511a-46a5-98f2-d252b6164ff6 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:46.427907 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=89,keystone:INSERT=11 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:18:46.430218 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f40859b-511a-46a5-98f2-d252b6164ff6 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:46.435529 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e717f94-2ebf-4df9-bcc5-d6035cd51e1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:46.446699 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e717f94-2ebf-4df9-bcc5-d6035cd51e1b admin admin] REQUEST_METHOD: `PUT` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:46.446699 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e717f94-2ebf-4df9-bcc5-d6035cd51e1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:46.446699 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e717f94-2ebf-4df9-bcc5-d6035cd51e1b admin admin] PATH_INFO: `/v3/projects/a5e956906b484196945c8deb97beaf5d/users/827bfe2d861146b3aa67cfe147dc16d3/roles/3255f77192524a16bc405173106871fc` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:46.450870 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e717f94-2ebf-4df9-bcc5-d6035cd51e1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5e956906b484196945c8deb97beaf5d, user_id=827bfe2d861146b3aa67cfe147dc16d3, role_id=3255f77192524a16bc405173106871fc)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:46.453511 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e717f94-2ebf-4df9-bcc5-d6035cd51e1b admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:46.490571 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 802/1592] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:18:46 2022] POST /identity/v3/projects => generated 365 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:18:46.490571 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8749a4a3-acbd-49f6-ac12-47f88cc52a8e admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:46.508672 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 791/1593] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:18:46 2022] PUT /identity/v3/projects/a5e956906b484196945c8deb97beaf5d/users/827bfe2d861146b3aa67cfe147dc16d3/roles/3255f77192524a16bc405173106871fc => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:18:46.510120 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e717f94-2ebf-4df9-bcc5-d6035cd51e1b admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:46.551099 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8749a4a3-acbd-49f6-ac12-47f88cc52a8e admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:46.574580 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e717f94-2ebf-4df9-bcc5-d6035cd51e1b admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:46.579328 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238cb5c3-eb92-48d4-b609-948a547907e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:46.579995 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238cb5c3-eb92-48d4-b609-948a547907e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:46.580276 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238cb5c3-eb92-48d4-b609-948a547907e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:46.580552 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238cb5c3-eb92-48d4-b609-948a547907e2 admin admin] PATH_INFO: `/v3/projects/a4754b68abd046f68f0f422c90a583f5/users/680dbbbc2e9249cb865ddc540461ac28/roles/3255f77192524a16bc405173106871fc` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:46.595616 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238cb5c3-eb92-48d4-b609-948a547907e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4754b68abd046f68f0f422c90a583f5, user_id=680dbbbc2e9249cb865ddc540461ac28, role_id=3255f77192524a16bc405173106871fc)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:46.598295 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238cb5c3-eb92-48d4-b609-948a547907e2 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:46.609470 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54082eec-46c9-44ac-81e4-bbb2d5619c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:46.612958 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54082eec-46c9-44ac-81e4-bbb2d5619c35 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:46.612958 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54082eec-46c9-44ac-81e4-bbb2d5619c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:46.612958 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54082eec-46c9-44ac-81e4-bbb2d5619c35 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:46.612958 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54082eec-46c9-44ac-81e4-bbb2d5619c35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:46.612958 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54082eec-46c9-44ac-81e4-bbb2d5619c35 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:46.623440 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 803/1594] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:18:46 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:18:46.624238 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54082eec-46c9-44ac-81e4-bbb2d5619c35 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:46.633802 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54082eec-46c9-44ac-81e4-bbb2d5619c35 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:46.638647 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa716b34-b825-4cae-83c4-00732189f224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:46.639294 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa716b34-b825-4cae-83c4-00732189f224 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:46.639578 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa716b34-b825-4cae-83c4-00732189f224 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:46.639856 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa716b34-b825-4cae-83c4-00732189f224 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:46.640509 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa716b34-b825-4cae-83c4-00732189f224 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:46.642175 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa716b34-b825-4cae-83c4-00732189f224 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:46.652109 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 792/1595] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:18:46 2022] PUT /identity/v3/projects/a4754b68abd046f68f0f422c90a583f5/users/680dbbbc2e9249cb865ddc540461ac28/roles/3255f77192524a16bc405173106871fc => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:18:46.653668 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 804/1596] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:18:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:18:46.653866 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa716b34-b825-4cae-83c4-00732189f224 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:46.656579 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238cb5c3-eb92-48d4-b609-948a547907e2 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:46.672172 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa716b34-b825-4cae-83c4-00732189f224 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:46.711233 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:18:46.725775 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238cb5c3-eb92-48d4-b609-948a547907e2 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:46.730504 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc8269e-d66f-4fa7-b771-a06d1dbad836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:46.731219 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc8269e-d66f-4fa7-b771-a06d1dbad836 admin admin] REQUEST_METHOD: `PUT` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:46.731995 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc8269e-d66f-4fa7-b771-a06d1dbad836 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:46.732273 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc8269e-d66f-4fa7-b771-a06d1dbad836 admin admin] PATH_INFO: `/v3/projects/e1082e3330ff43b58852314736751cc6/users/37446da6d8d24fd99257767814d5f4d9/roles/3255f77192524a16bc405173106871fc` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:46.732651 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca37862b-4290-4e58-b9af-30aee7297c51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:46.733264 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca37862b-4290-4e58-b9af-30aee7297c51 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:46.733659 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca37862b-4290-4e58-b9af-30aee7297c51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:46.733942 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca37862b-4290-4e58-b9af-30aee7297c51 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:46.734627 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca37862b-4290-4e58-b9af-30aee7297c51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:46.736385 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca37862b-4290-4e58-b9af-30aee7297c51 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:46.746460 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 805/1597] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:18:46 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:18:46.746621 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc8269e-d66f-4fa7-b771-a06d1dbad836 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1082e3330ff43b58852314736751cc6, user_id=37446da6d8d24fd99257767814d5f4d9, role_id=3255f77192524a16bc405173106871fc)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:46.747317 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca37862b-4290-4e58-b9af-30aee7297c51 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:46.749621 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc8269e-d66f-4fa7-b771-a06d1dbad836 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:46.756960 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca37862b-4290-4e58-b9af-30aee7297c51 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:46.761784 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f550ef-289f-40da-9f80-993d0d70fb93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:46.762432 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f550ef-289f-40da-9f80-993d0d70fb93 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:46.762716 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f550ef-289f-40da-9f80-993d0d70fb93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:46.762992 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f550ef-289f-40da-9f80-993d0d70fb93 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:46.763622 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f550ef-289f-40da-9f80-993d0d70fb93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:46.765312 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f550ef-289f-40da-9f80-993d0d70fb93 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:46.776473 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 806/1598] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:18:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:18:46.779407 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f550ef-289f-40da-9f80-993d0d70fb93 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:46.789759 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f550ef-289f-40da-9f80-993d0d70fb93 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:46.795197 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894b20a1-5e28-494b-93ba-6c62db82bbf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:46.795970 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894b20a1-5e28-494b-93ba-6c62db82bbf2 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:46.796261 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894b20a1-5e28-494b-93ba-6c62db82bbf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:46.796539 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894b20a1-5e28-494b-93ba-6c62db82bbf2 admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:46.797165 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894b20a1-5e28-494b-93ba-6c62db82bbf2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:46.799205 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894b20a1-5e28-494b-93ba-6c62db82bbf2 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:46.810391 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:18:46.863673 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 793/1599] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:18:46 2022] PUT /identity/v3/projects/e1082e3330ff43b58852314736751cc6/users/37446da6d8d24fd99257767814d5f4d9/roles/3255f77192524a16bc405173106871fc => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:18:46.865072 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fae141-e440-4a12-8967-d9ba0cf1251f None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:46.865394 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fae141-e440-4a12-8967-d9ba0cf1251f None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:46.865671 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fae141-e440-4a12-8967-d9ba0cf1251f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:46.884460 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-d7fae141-e440-4a12-8967-d9ba0cf1251f None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:18:46.885748 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-d7fae141-e440-4a12-8967-d9ba0cf1251f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:18:46.888930 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 807/1600] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:18:46 2022] POST /identity/v3/users => generated 289 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:18:46.888930 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894b20a1-5e28-494b-93ba-6c62db82bbf2 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:46.896588 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894b20a1-5e28-494b-93ba-6c62db82bbf2 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:46.946708 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 794/1601] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:18:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:18:46.947814 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fae141-e440-4a12-8967-d9ba0cf1251f None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:46.957114 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb8dff7-7257-4ae3-a54e-079d5155d561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:46.957671 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fae141-e440-4a12-8967-d9ba0cf1251f None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:46.957959 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb8dff7-7257-4ae3-a54e-079d5155d561 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:46.958343 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb8dff7-7257-4ae3-a54e-079d5155d561 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:46.958694 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb8dff7-7257-4ae3-a54e-079d5155d561 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:46.959414 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb8dff7-7257-4ae3-a54e-079d5155d561 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:46.961105 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb8dff7-7257-4ae3-a54e-079d5155d561 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:46.962194 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e567cf-8fe3-425c-a327-78611ee95cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:46.962841 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e567cf-8fe3-425c-a327-78611ee95cb2 admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:46.963112 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e567cf-8fe3-425c-a327-78611ee95cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:46.963416 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e567cf-8fe3-425c-a327-78611ee95cb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:46.964064 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e567cf-8fe3-425c-a327-78611ee95cb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:46.966141 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e567cf-8fe3-425c-a327-78611ee95cb2 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:46.975209 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 808/1602] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:18:46 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:18:46.975209 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b08cf0-e416-4b41-9c97-b2a2eb0b8e1d None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:46.975209 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b08cf0-e416-4b41-9c97-b2a2eb0b8e1d None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:46.975209 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b08cf0-e416-4b41-9c97-b2a2eb0b8e1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:46.976841 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 795/1603] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:18:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:18:46.977592 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e567cf-8fe3-425c-a327-78611ee95cb2 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:46.988008 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e567cf-8fe3-425c-a327-78611ee95cb2 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:46.992794 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ead8b1b-7329-4b35-b788-d24dd40fcfc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:46.993446 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ead8b1b-7329-4b35-b788-d24dd40fcfc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:46.993722 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ead8b1b-7329-4b35-b788-d24dd40fcfc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:46.993999 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ead8b1b-7329-4b35-b788-d24dd40fcfc8 admin admin] PATH_INFO: `/v3/projects/d8e6240beed94fc38a7f1b85c7fad39c/users/83e90afadb164c82bc93c47099c14460/roles/3255f77192524a16bc405173106871fc` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:47.004394 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-e6b08cf0-e416-4b41-9c97-b2a2eb0b8e1d None None] MFA Rules not processed for user `680dbbbc2e9249cb865ddc540461ac28`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:18:47.007089 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-e6b08cf0-e416-4b41-9c97-b2a2eb0b8e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:18:47.008728 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ead8b1b-7329-4b35-b788-d24dd40fcfc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8e6240beed94fc38a7f1b85c7fad39c, user_id=83e90afadb164c82bc93c47099c14460, role_id=3255f77192524a16bc405173106871fc)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:47.011436 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ead8b1b-7329-4b35-b788-d24dd40fcfc8 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:47.072209 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 809/1604] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:18:46 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:18:47.073612 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b08cf0-e416-4b41-9c97-b2a2eb0b8e1d None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:47.083184 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 796/1605] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:18:46 2022] PUT /identity/v3/projects/d8e6240beed94fc38a7f1b85c7fad39c/users/83e90afadb164c82bc93c47099c14460/roles/3255f77192524a16bc405173106871fc => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:18:47.084785 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898a9cf9-e740-4994-a719-b541bc188060 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:47.084909 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b08cf0-e416-4b41-9c97-b2a2eb0b8e1d None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:47.085288 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898a9cf9-e740-4994-a719-b541bc188060 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:47.085665 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898a9cf9-e740-4994-a719-b541bc188060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:47.116214 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-898a9cf9-e740-4994-a719-b541bc188060 None None] MFA Rules not processed for user `37446da6d8d24fd99257767814d5f4d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:18:47.117429 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-898a9cf9-e740-4994-a719-b541bc188060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:18:47.149166 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f389d556-625d-46c5-bb18-e15cfd82f856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:47.149840 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f389d556-625d-46c5-bb18-e15cfd82f856 admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:47.150124 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f389d556-625d-46c5-bb18-e15cfd82f856 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:47.150449 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f389d556-625d-46c5-bb18-e15cfd82f856 admin admin] PATH_INFO: `/v3/projects/48be9cd602794803911476ce3390dd55/users/539fbb2860474b69883aa3d5e88bd4f0/roles/3255f77192524a16bc405173106871fc` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:47.166814 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f389d556-625d-46c5-bb18-e15cfd82f856 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48be9cd602794803911476ce3390dd55, user_id=539fbb2860474b69883aa3d5e88bd4f0, role_id=3255f77192524a16bc405173106871fc)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:47.169499 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f389d556-625d-46c5-bb18-e15cfd82f856 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:47.184787 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 797/1606] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:18:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:18:47.185844 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898a9cf9-e740-4994-a719-b541bc188060 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:47.200636 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898a9cf9-e740-4994-a719-b541bc188060 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:47.205621 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2704f55b-241b-406d-baff-5688646ee6c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:47.206233 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2704f55b-241b-406d-baff-5688646ee6c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:47.206558 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2704f55b-241b-406d-baff-5688646ee6c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:47.206832 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2704f55b-241b-406d-baff-5688646ee6c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:47.207536 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2704f55b-241b-406d-baff-5688646ee6c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:47.209146 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2704f55b-241b-406d-baff-5688646ee6c6 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:47.223516 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 798/1607] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:18:47 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:18:47.223516 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2704f55b-241b-406d-baff-5688646ee6c6 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:47.234166 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:18:47.251883 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 810/1608] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:18:47 2022] PUT /identity/v3/projects/48be9cd602794803911476ce3390dd55/users/539fbb2860474b69883aa3d5e88bd4f0/roles/3255f77192524a16bc405173106871fc => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:18:47.252706 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f389d556-625d-46c5-bb18-e15cfd82f856 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:47.265227 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:18:47.277160 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2704f55b-241b-406d-baff-5688646ee6c6 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:47.320737 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f389d556-625d-46c5-bb18-e15cfd82f856 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:47.326635 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb50039-eaab-4db2-8cf5-34c1e64c84d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:47.328372 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb50039-eaab-4db2-8cf5-34c1e64c84d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:47.328498 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b0dd2f-7e31-41c1-a113-7812bb391759 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:47.328624 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb50039-eaab-4db2-8cf5-34c1e64c84d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:47.328624 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb50039-eaab-4db2-8cf5-34c1e64c84d9 admin admin] PATH_INFO: `/v3/projects/fe809c1bb51c4f8c9380c1ada3643ce9/users/30737ce2040a4b5aac1c6c587abb9a81/roles/3255f77192524a16bc405173106871fc` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:47.328784 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b0dd2f-7e31-41c1-a113-7812bb391759 service neutron] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:47.328784 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b0dd2f-7e31-41c1-a113-7812bb391759 service neutron] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:47.328784 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b0dd2f-7e31-41c1-a113-7812bb391759 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:47.337578 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b0dd2f-7e31-41c1-a113-7812bb391759 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:47.339030 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b0dd2f-7e31-41c1-a113-7812bb391759 service neutron] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:47.343864 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb50039-eaab-4db2-8cf5-34c1e64c84d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe809c1bb51c4f8c9380c1ada3643ce9, user_id=30737ce2040a4b5aac1c6c587abb9a81, role_id=3255f77192524a16bc405173106871fc)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:47.346653 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb50039-eaab-4db2-8cf5-34c1e64c84d9 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:47.351540 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 799/1609] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:18:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:18:47.351540 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b0dd2f-7e31-41c1-a113-7812bb391759 service neutron] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:47.358684 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b0dd2f-7e31-41c1-a113-7812bb391759 service neutron] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:47.363764 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a2fd94-7410-43b6-af9e-58cbce6115b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:47.364420 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a2fd94-7410-43b6-af9e-58cbce6115b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:47.364705 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a2fd94-7410-43b6-af9e-58cbce6115b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:47.364981 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a2fd94-7410-43b6-af9e-58cbce6115b8 admin admin] PATH_INFO: `/v3/projects/d0928e519ec247b988a254d7918cbfed/users/c7dc5019b26a4a8abf6436be3f1e56af/roles/3255f77192524a16bc405173106871fc` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:47.380414 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a2fd94-7410-43b6-af9e-58cbce6115b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0928e519ec247b988a254d7918cbfed, user_id=c7dc5019b26a4a8abf6436be3f1e56af, role_id=3255f77192524a16bc405173106871fc)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:47.383102 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a2fd94-7410-43b6-af9e-58cbce6115b8 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:47.417479 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 811/1610] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:18:47 2022] PUT /identity/v3/projects/fe809c1bb51c4f8c9380c1ada3643ce9/users/30737ce2040a4b5aac1c6c587abb9a81/roles/3255f77192524a16bc405173106871fc => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:18:47.422007 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c302ea1-2633-4f45-a280-bb89131620ca None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:47.422007 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c302ea1-2633-4f45-a280-bb89131620ca None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:47.422007 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c302ea1-2633-4f45-a280-bb89131620ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:47.441378 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 800/1611] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:18:47 2022] PUT /identity/v3/projects/d0928e519ec247b988a254d7918cbfed/users/c7dc5019b26a4a8abf6436be3f1e56af/roles/3255f77192524a16bc405173106871fc => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:18:47.441378 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e230c07c-3e80-4022-a42c-3220b01b920a None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:47.441378 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e230c07c-3e80-4022-a42c-3220b01b920a None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:47.441378 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e230c07c-3e80-4022-a42c-3220b01b920a None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:47.441806 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 801/1612] 10.222.0.58 () {54 vars in 962 bytes} [Thu Aug 25 03:18:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:18:47.442849 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090fba79-de02-4b81-b186-5f708c41f5a9 None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:47.443175 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090fba79-de02-4b81-b186-5f708c41f5a9 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:47.443382 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090fba79-de02-4b81-b186-5f708c41f5a9 None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:47.444872 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 802/1613] 10.222.0.58 () {54 vars in 962 bytes} [Thu Aug 25 03:18:47 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:18:47.445558 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090fba79-de02-4b81-b186-5f708c41f5a9 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:47.446339 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-2c302ea1-2633-4f45-a280-bb89131620ca None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:18:47.447713 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-2c302ea1-2633-4f45-a280-bb89131620ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:18:47.517250 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 812/1614] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:18:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:18:47.519182 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08371bb5-68ff-4e25-b99d-b8743f170f6c None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:47.519633 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08371bb5-68ff-4e25-b99d-b8743f170f6c None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:47.520051 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08371bb5-68ff-4e25-b99d-b8743f170f6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:47.523191 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090fba79-de02-4b81-b186-5f708c41f5a9 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:47.530196 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699819e7-53e6-458b-b312-3be0d76c0fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:47.530866 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699819e7-53e6-458b-b312-3be0d76c0fbc admin admin] REQUEST_METHOD: `PUT` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:47.531187 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699819e7-53e6-458b-b312-3be0d76c0fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:47.531420 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699819e7-53e6-458b-b312-3be0d76c0fbc admin admin] PATH_INFO: `/v3/projects/42d84da7a4154e4c8075f60b563cbfe5/users/b5cbd421afee4e08928dacc23e119736/roles/62f19ea272b447ccaab1576cd9bf4fd9` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:47.546023 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699819e7-53e6-458b-b312-3be0d76c0fbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=42d84da7a4154e4c8075f60b563cbfe5, user_id=b5cbd421afee4e08928dacc23e119736, role_id=62f19ea272b447ccaab1576cd9bf4fd9)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:47.548775 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-08371bb5-68ff-4e25-b99d-b8743f170f6c None None] MFA Rules not processed for user `539fbb2860474b69883aa3d5e88bd4f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:18:47.548928 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699819e7-53e6-458b-b312-3be0d76c0fbc admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:47.550130 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-08371bb5-68ff-4e25-b99d-b8743f170f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:18:47.610831 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 803/1615] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:18:47 2022] PUT /identity/v3/projects/42d84da7a4154e4c8075f60b563cbfe5/users/b5cbd421afee4e08928dacc23e119736/roles/62f19ea272b447ccaab1576cd9bf4fd9 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:18:47.612429 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a48e0c4-8215-4231-a039-33ffdf6fb264 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:47.612982 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a48e0c4-8215-4231-a039-33ffdf6fb264 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:47.613470 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a48e0c4-8215-4231-a039-33ffdf6fb264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:47.616451 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 813/1616] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:18:47 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:18:47.618186 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67448995-ff7c-4368-af0f-52c07072a5f9 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:47.618186 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67448995-ff7c-4368-af0f-52c07072a5f9 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:47.619307 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67448995-ff7c-4368-af0f-52c07072a5f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:47.643508 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-67448995-ff7c-4368-af0f-52c07072a5f9 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:18:47.643508 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-67448995-ff7c-4368-af0f-52c07072a5f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:18:47.657756 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-5a48e0c4-8215-4231-a039-33ffdf6fb264 None None] MFA Rules not processed for user `30737ce2040a4b5aac1c6c587abb9a81`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:18:47.659429 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:18:47.659922 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-5a48e0c4-8215-4231-a039-33ffdf6fb264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:18:47.683950 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:18:47.710706 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 814/1617] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:18:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:18:47.712251 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684bb8dd-d67d-4936-b0ac-5b69c0e806f1 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:47.712639 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684bb8dd-d67d-4936-b0ac-5b69c0e806f1 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:47.713029 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684bb8dd-d67d-4936-b0ac-5b69c0e806f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:47.743873 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 804/1618] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:18:47 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:18:47.745746 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c321920d-9b4b-4fb1-a000-8095f9242db0 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:47.746048 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c321920d-9b4b-4fb1-a000-8095f9242db0 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:47.746407 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c321920d-9b4b-4fb1-a000-8095f9242db0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:47.752435 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-684bb8dd-d67d-4936-b0ac-5b69c0e806f1 None None] MFA Rules not processed for user `0cd171bf4df94f8fab397e42835feeab`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:18:47.753829 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-684bb8dd-d67d-4936-b0ac-5b69c0e806f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:18:47.766238 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-c321920d-9b4b-4fb1-a000-8095f9242db0 None None] MFA Rules not processed for user `0cd171bf4df94f8fab397e42835feeab`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:18:47.767471 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-c321920d-9b4b-4fb1-a000-8095f9242db0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:18:47.828085 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 815/1619] 10.222.0.58 () {58 vars in 1059 bytes} [Thu Aug 25 03:18:47 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:18:47.829113 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684bb8dd-d67d-4936-b0ac-5b69c0e806f1 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:47.845786 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 805/1620] 10.222.0.58 () {58 vars in 1059 bytes} [Thu Aug 25 03:18:47 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:18:47.847604 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c321920d-9b4b-4fb1-a000-8095f9242db0 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:47.865273 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c321920d-9b4b-4fb1-a000-8095f9242db0 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:47.876020 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb70a68-e513-4351-9d8c-cc06efffb0e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:47.877026 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb70a68-e513-4351-9d8c-cc06efffb0e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:47.877430 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb70a68-e513-4351-9d8c-cc06efffb0e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:47.877793 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb70a68-e513-4351-9d8c-cc06efffb0e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:47.881749 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb70a68-e513-4351-9d8c-cc06efffb0e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:47.884825 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb70a68-e513-4351-9d8c-cc06efffb0e9 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:47.896710 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684bb8dd-d67d-4936-b0ac-5b69c0e806f1 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:47.897744 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 806/1621] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:18:47 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:18:47.898698 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb70a68-e513-4351-9d8c-cc06efffb0e9 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:47.900724 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d375b2-3232-4a08-8291-753ad080120a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:47.901433 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d375b2-3232-4a08-8291-753ad080120a service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:47.902083 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d375b2-3232-4a08-8291-753ad080120a service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:47.902387 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d375b2-3232-4a08-8291-753ad080120a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:47.917979 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d375b2-3232-4a08-8291-753ad080120a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:47.920196 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d375b2-3232-4a08-8291-753ad080120a service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:47.932971 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 816/1622] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:18:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:18:47.934880 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d375b2-3232-4a08-8291-753ad080120a service neutron] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:47.936102 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-46d375b2-3232-4a08-8291-753ad080120a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:18:47.951464 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d375b2-3232-4a08-8291-753ad080120a service neutron] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:47.956323 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6c6f49-7ae0-4549-8765-3254dcc62e33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:47.956940 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6c6f49-7ae0-4549-8765-3254dcc62e33 service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:47.957273 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6c6f49-7ae0-4549-8765-3254dcc62e33 service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:47.957546 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6c6f49-7ae0-4549-8765-3254dcc62e33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:47.968996 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6c6f49-7ae0-4549-8765-3254dcc62e33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:47.970475 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6c6f49-7ae0-4549-8765-3254dcc62e33 service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:47.980728 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 817/1623] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:18:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:18:47.982124 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6c6f49-7ae0-4549-8765-3254dcc62e33 service neutron] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:47.991478 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb70a68-e513-4351-9d8c-cc06efffb0e9 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:47.992493 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6c6f49-7ae0-4549-8765-3254dcc62e33 service neutron] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:47.996179 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c1f30a-b2fd-4056-b2aa-1b677faef1de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60193ba9e1c14df3bc9eebcc6f28f2d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:47.997473 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c1f30a-b2fd-4056-b2aa-1b677faef1de service cinder] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:47.997760 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c1f30a-b2fd-4056-b2aa-1b677faef1de service cinder] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:47.997885 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c73e1e3-2794-4138-b028-71cf524a1771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:47.998041 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c1f30a-b2fd-4056-b2aa-1b677faef1de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:47.998416 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c73e1e3-2794-4138-b028-71cf524a1771 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:47.998743 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c73e1e3-2794-4138-b028-71cf524a1771 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:47.999019 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c73e1e3-2794-4138-b028-71cf524a1771 admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:47.999650 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c73e1e3-2794-4138-b028-71cf524a1771 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:48.001564 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c73e1e3-2794-4138-b028-71cf524a1771 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:48.060119 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:18:48.063346 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c1f30a-b2fd-4056-b2aa-1b677faef1de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:48.064967 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c1f30a-b2fd-4056-b2aa-1b677faef1de service cinder] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:48.075659 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 807/1624] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:18:47 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:18:48.076641 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c1f30a-b2fd-4056-b2aa-1b677faef1de service cinder] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:48.090142 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c1f30a-b2fd-4056-b2aa-1b677faef1de service cinder] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:48.104816 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 818/1625] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:18:47 2022] POST /identity/v3/projects => generated 369 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:18:48.104816 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c73e1e3-2794-4138-b028-71cf524a1771 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:48.112575 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:18:48.115287 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34d3eb3-1930-446a-8006-4bca86759afe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cd171bf4df94f8fab397e42835feeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:48.115981 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34d3eb3-1930-446a-8006-4bca86759afe service nova] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:48.116492 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34d3eb3-1930-446a-8006-4bca86759afe service nova] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:48.120038 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34d3eb3-1930-446a-8006-4bca86759afe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:48.202026 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34d3eb3-1930-446a-8006-4bca86759afe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:48.203353 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34d3eb3-1930-446a-8006-4bca86759afe service nova] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:48.215111 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 808/1626] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:18:48 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:18:48.216750 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34d3eb3-1930-446a-8006-4bca86759afe service nova] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:48.220759 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c73e1e3-2794-4138-b028-71cf524a1771 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:48.228915 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34d3eb3-1930-446a-8006-4bca86759afe service nova] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:48.238047 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a9a21e-aefa-462b-9fad-1a2c3d8bcde2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cd171bf4df94f8fab397e42835feeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:48.239076 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a9a21e-aefa-462b-9fad-1a2c3d8bcde2 service nova] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:48.239434 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a9a21e-aefa-462b-9fad-1a2c3d8bcde2 service nova] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:48.240724 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a9a21e-aefa-462b-9fad-1a2c3d8bcde2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:48.302827 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1685f01-e825-4fdf-9cc2-1ae1e896588c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:48.303789 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1685f01-e825-4fdf-9cc2-1ae1e896588c admin admin] REQUEST_METHOD: `PUT` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:48.304083 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1685f01-e825-4fdf-9cc2-1ae1e896588c admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:48.304363 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1685f01-e825-4fdf-9cc2-1ae1e896588c admin admin] PATH_INFO: `/v3/projects/42d84da7a4154e4c8075f60b563cbfe5/users/b5cbd421afee4e08928dacc23e119736/roles/62f19ea272b447ccaab1576cd9bf4fd9` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:48.307170 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1685f01-e825-4fdf-9cc2-1ae1e896588c admin admin] RBAC: Authorizing `identity:create_grant(project_id=42d84da7a4154e4c8075f60b563cbfe5, user_id=b5cbd421afee4e08928dacc23e119736, role_id=62f19ea272b447ccaab1576cd9bf4fd9)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:48.310877 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1685f01-e825-4fdf-9cc2-1ae1e896588c admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:48.326653 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 809/1627] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:18:48 2022] PUT /identity/v3/projects/42d84da7a4154e4c8075f60b563cbfe5/users/b5cbd421afee4e08928dacc23e119736/roles/62f19ea272b447ccaab1576cd9bf4fd9 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:18:48.327553 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1685f01-e825-4fdf-9cc2-1ae1e896588c admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:48.368590 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a9a21e-aefa-462b-9fad-1a2c3d8bcde2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:48.371781 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a9a21e-aefa-462b-9fad-1a2c3d8bcde2 service nova] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:48.393994 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 819/1628] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:18:48 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:18:48.397500 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c2448e-9ed9-4348-811e-d05dc4a0b3c2 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:48.397986 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c2448e-9ed9-4348-811e-d05dc4a0b3c2 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:48.399511 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c2448e-9ed9-4348-811e-d05dc4a0b3c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:48.412708 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1685f01-e825-4fdf-9cc2-1ae1e896588c admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:48.419789 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1e5f28-ddc3-4297-ab73-ceeb2d5699b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:48.422076 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1e5f28-ddc3-4297-ab73-ceeb2d5699b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:48.422076 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1e5f28-ddc3-4297-ab73-ceeb2d5699b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:48.422076 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1e5f28-ddc3-4297-ab73-ceeb2d5699b9 admin admin] PATH_INFO: `/v3/users` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:48.422076 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1e5f28-ddc3-4297-ab73-ceeb2d5699b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:48.424183 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1e5f28-ddc3-4297-ab73-ceeb2d5699b9 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:48.461331 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-12c2448e-9ed9-4348-811e-d05dc4a0b3c2 None None] MFA Rules not processed for user `b5cbd421afee4e08928dacc23e119736`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:18:48.462943 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-12c2448e-9ed9-4348-811e-d05dc4a0b3c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:18:48.530229 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 810/1629] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:18:48 2022] POST /identity/v3/users => generated 291 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:18:48.533836 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1e5f28-ddc3-4297-ab73-ceeb2d5699b9 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:48.568724 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1e5f28-ddc3-4297-ab73-ceeb2d5699b9 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:48.592710 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157285e5-53b5-4d5c-87a1-9901f5985acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:48.595633 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157285e5-53b5-4d5c-87a1-9901f5985acb admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:48.597006 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157285e5-53b5-4d5c-87a1-9901f5985acb admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:48.597291 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157285e5-53b5-4d5c-87a1-9901f5985acb admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:48.618509 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157285e5-53b5-4d5c-87a1-9901f5985acb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:48.623649 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157285e5-53b5-4d5c-87a1-9901f5985acb admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:48.627133 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 820/1630] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:18:48 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:18:48.647692 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c2448e-9ed9-4348-811e-d05dc4a0b3c2 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:48.653989 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 811/1631] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:18:48 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:18:48.661212 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157285e5-53b5-4d5c-87a1-9901f5985acb admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:48.688305 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157285e5-53b5-4d5c-87a1-9901f5985acb admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:48.701700 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabca476-fd50-4741-8cd9-4441c3b4bf40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:48.703478 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabca476-fd50-4741-8cd9-4441c3b4bf40 admin admin] REQUEST_METHOD: `PUT` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:48.703800 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabca476-fd50-4741-8cd9-4441c3b4bf40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:48.704085 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabca476-fd50-4741-8cd9-4441c3b4bf40 admin admin] PATH_INFO: `/v3/projects/4a5aa2842b9a45fa991e41b7764abe14/users/1a186753581e40f3a51356637f0bd358/roles/62f19ea272b447ccaab1576cd9bf4fd9` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:48.707283 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:18:48.707843 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:18:48.726193 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabca476-fd50-4741-8cd9-4441c3b4bf40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a5aa2842b9a45fa991e41b7764abe14, user_id=1a186753581e40f3a51356637f0bd358, role_id=62f19ea272b447ccaab1576cd9bf4fd9)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:48.731136 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabca476-fd50-4741-8cd9-4441c3b4bf40 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:48.752834 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c2448e-9ed9-4348-811e-d05dc4a0b3c2 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:48.757491 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf422853-76b6-4f7b-b901-d0935f1ff8ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60193ba9e1c14df3bc9eebcc6f28f2d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:48.758226 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf422853-76b6-4f7b-b901-d0935f1ff8ce service cinder] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:48.758621 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf422853-76b6-4f7b-b901-d0935f1ff8ce service cinder] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:48.759261 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf422853-76b6-4f7b-b901-d0935f1ff8ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:48.770890 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf422853-76b6-4f7b-b901-d0935f1ff8ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:48.772620 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf422853-76b6-4f7b-b901-d0935f1ff8ce service cinder] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:48.806402 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 821/1632] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:18:48 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:18:48.808341 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b440fc1-46f6-4c7e-ab4f-027a45215b3b None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:48.808627 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b440fc1-46f6-4c7e-ab4f-027a45215b3b None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:48.808901 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b440fc1-46f6-4c7e-ab4f-027a45215b3b None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:48.810882 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 822/1633] 10.222.0.58 () {54 vars in 912 bytes} [Thu Aug 25 03:18:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:18:48.820514 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bd8eef-9aab-4f68-9d98-943f891fe77c None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:48.820834 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bd8eef-9aab-4f68-9d98-943f891fe77c None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:48.821112 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bd8eef-9aab-4f68-9d98-943f891fe77c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:48.826960 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 812/1634] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:18:48 2022] PUT /identity/v3/projects/4a5aa2842b9a45fa991e41b7764abe14/users/1a186753581e40f3a51356637f0bd358/roles/62f19ea272b447ccaab1576cd9bf4fd9 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:18:48.830825 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabca476-fd50-4741-8cd9-4441c3b4bf40 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:48.865279 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-f0bd8eef-9aab-4f68-9d98-943f891fe77c None None] MFA Rules not processed for user `0cd171bf4df94f8fab397e42835feeab`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:18:48.871057 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-f0bd8eef-9aab-4f68-9d98-943f891fe77c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:18:48.945309 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabca476-fd50-4741-8cd9-4441c3b4bf40 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:48.953521 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7858ebcf-fc53-4811-a3b1-70507e9e630a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:48.955005 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7858ebcf-fc53-4811-a3b1-70507e9e630a admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:48.962198 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7858ebcf-fc53-4811-a3b1-70507e9e630a admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:48.962198 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7858ebcf-fc53-4811-a3b1-70507e9e630a admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:48.962198 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7858ebcf-fc53-4811-a3b1-70507e9e630a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:48.962198 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7858ebcf-fc53-4811-a3b1-70507e9e630a admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:48.974118 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 813/1635] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:18:48 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 25 03:18:48.987301 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7858ebcf-fc53-4811-a3b1-70507e9e630a admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:49.023663 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7858ebcf-fc53-4811-a3b1-70507e9e630a admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:49.040617 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89fe494-ef99-4746-9904-51f3dc23cd1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:49.041314 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89fe494-ef99-4746-9904-51f3dc23cd1d admin admin] REQUEST_METHOD: `PUT` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:49.041600 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89fe494-ef99-4746-9904-51f3dc23cd1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:49.042476 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89fe494-ef99-4746-9904-51f3dc23cd1d admin admin] PATH_INFO: `/v3/projects/4a5aa2842b9a45fa991e41b7764abe14/users/1a186753581e40f3a51356637f0bd358/roles/62f19ea272b447ccaab1576cd9bf4fd9` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:49.046129 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89fe494-ef99-4746-9904-51f3dc23cd1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a5aa2842b9a45fa991e41b7764abe14, user_id=1a186753581e40f3a51356637f0bd358, role_id=62f19ea272b447ccaab1576cd9bf4fd9)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:49.049700 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89fe494-ef99-4746-9904-51f3dc23cd1d admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:49.073027 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 814/1636] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:18:48 2022] PUT /identity/v3/projects/4a5aa2842b9a45fa991e41b7764abe14/users/1a186753581e40f3a51356637f0bd358/roles/62f19ea272b447ccaab1576cd9bf4fd9 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:18:49.076342 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c879a8-691f-4a55-a763-f18fa5fe6d00 None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:49.076672 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c879a8-691f-4a55-a763-f18fa5fe6d00 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:49.076922 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c879a8-691f-4a55-a763-f18fa5fe6d00 None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:49.078377 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 815/1637] 10.222.0.58 () {54 vars in 912 bytes} [Thu Aug 25 03:18:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:18:49.083967 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9215c76-492c-4d1a-81bc-df8bc5dae90b None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:49.084263 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9215c76-492c-4d1a-81bc-df8bc5dae90b None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:49.085395 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9215c76-492c-4d1a-81bc-df8bc5dae90b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:49.094376 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 823/1638] 10.222.0.58 () {58 vars in 1009 bytes} [Thu Aug 25 03:18:48 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:18:49.095710 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a1d401-ef4a-4f13-9ce2-f9ac177a6af7 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:49.096103 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a1d401-ef4a-4f13-9ce2-f9ac177a6af7 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:49.097448 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a1d401-ef4a-4f13-9ce2-f9ac177a6af7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:49.139696 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-34a1d401-ef4a-4f13-9ce2-f9ac177a6af7 None None] MFA Rules not processed for user `0cd171bf4df94f8fab397e42835feeab`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:18:49.140944 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-34a1d401-ef4a-4f13-9ce2-f9ac177a6af7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:18:49.152156 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-a9215c76-492c-4d1a-81bc-df8bc5dae90b None None] MFA Rules not processed for user `1a186753581e40f3a51356637f0bd358`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:18:49.155625 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-a9215c76-492c-4d1a-81bc-df8bc5dae90b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:18:49.270566 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 824/1639] 10.222.0.58 () {58 vars in 1009 bytes} [Thu Aug 25 03:18:49 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:18:49.271824 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a1d401-ef4a-4f13-9ce2-f9ac177a6af7 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:49.285453 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 816/1640] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:18:49 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:18:49.353094 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a1d401-ef4a-4f13-9ce2-f9ac177a6af7 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:49.376632 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316e70da-1c0b-4b18-968b-42ce7b30faa6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60193ba9e1c14df3bc9eebcc6f28f2d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:49.377264 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316e70da-1c0b-4b18-968b-42ce7b30faa6 service cinder] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:49.377872 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316e70da-1c0b-4b18-968b-42ce7b30faa6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:49.378156 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316e70da-1c0b-4b18-968b-42ce7b30faa6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:49.394522 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:18:49.399856 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316e70da-1c0b-4b18-968b-42ce7b30faa6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:49.405787 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316e70da-1c0b-4b18-968b-42ce7b30faa6 service cinder] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:49.420485 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 825/1641] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:18:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:18:49.443740 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9215c76-492c-4d1a-81bc-df8bc5dae90b None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:49.455356 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9215c76-492c-4d1a-81bc-df8bc5dae90b None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:49.460031 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea56b3f-0f1a-40ef-b794-407514594787 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60193ba9e1c14df3bc9eebcc6f28f2d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:49.464159 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea56b3f-0f1a-40ef-b794-407514594787 service cinder] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:49.464159 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea56b3f-0f1a-40ef-b794-407514594787 service cinder] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:49.464159 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea56b3f-0f1a-40ef-b794-407514594787 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:49.475894 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea56b3f-0f1a-40ef-b794-407514594787 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:49.477792 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea56b3f-0f1a-40ef-b794-407514594787 service cinder] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:49.493557 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 817/1642] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:18:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:18:49.586088 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316e70da-1c0b-4b18-968b-42ce7b30faa6 service cinder] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:49.598367 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316e70da-1c0b-4b18-968b-42ce7b30faa6 service cinder] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:49.605665 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec4f307-f4fe-4104-86b0-f2bc8a9979d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60193ba9e1c14df3bc9eebcc6f28f2d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:49.607016 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec4f307-f4fe-4104-86b0-f2bc8a9979d4 service cinder] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:49.607297 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec4f307-f4fe-4104-86b0-f2bc8a9979d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:49.607572 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec4f307-f4fe-4104-86b0-f2bc8a9979d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:49.633523 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec4f307-f4fe-4104-86b0-f2bc8a9979d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:49.635693 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec4f307-f4fe-4104-86b0-f2bc8a9979d4 service cinder] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:49.664238 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 826/1643] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:18:49 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:18:50.258217 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea56b3f-0f1a-40ef-b794-407514594787 service cinder] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:50.345079 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:18:50.384535 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea56b3f-0f1a-40ef-b794-407514594787 service cinder] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:50.390727 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca77afe-6789-433a-a418-37456dc27b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:50.391866 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca77afe-6789-433a-a418-37456dc27b6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:50.392427 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca77afe-6789-433a-a418-37456dc27b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:50.392977 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca77afe-6789-433a-a418-37456dc27b6d admin admin] PATH_INFO: `/v3/users/37446da6d8d24fd99257767814d5f4d9` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:50.394861 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca77afe-6789-433a-a418-37456dc27b6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=37446da6d8d24fd99257767814d5f4d9)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:50.396095 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: WARNING py.warnings [None req-eca77afe-6789-433a-a418-37456dc27b6d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 25 03:18:50.396095 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: warnings.warn(msg) Aug 25 03:18:50.396095 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: Aug 25 03:18:50.398666 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca77afe-6789-433a-a418-37456dc27b6d admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:50.740793 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-eca77afe-6789-433a-a418-37456dc27b6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37446da6d8d24fd99257767814d5f4d9'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:18:50.847911 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-eca77afe-6789-433a-a418-37456dc27b6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37446da6d8d24fd99257767814d5f4d9'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:18:50.851058 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-eca77afe-6789-433a-a418-37456dc27b6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37446da6d8d24fd99257767814d5f4d9'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:18:50.851058 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-eca77afe-6789-433a-a418-37456dc27b6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37446da6d8d24fd99257767814d5f4d9'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:18:50.901411 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-eca77afe-6789-433a-a418-37456dc27b6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37446da6d8d24fd99257767814d5f4d9'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:18:50.928961 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 818/1644] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:18:50 2022] DELETE /identity/v3/users/37446da6d8d24fd99257767814d5f4d9 => generated 0 bytes in 671 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:18:50.935113 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c3a202-44cd-46db-b92a-135992ebab33 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:50.935691 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c3a202-44cd-46db-b92a-135992ebab33 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:50.935788 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c3a202-44cd-46db-b92a-135992ebab33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:50.959340 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-55c3a202-44cd-46db-b92a-135992ebab33 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:18:50.960893 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-55c3a202-44cd-46db-b92a-135992ebab33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:18:51.043191 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 827/1645] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:18:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:18:51.053927 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca77afe-6789-433a-a418-37456dc27b6d admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:51.055493 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-eca77afe-6789-433a-a418-37456dc27b6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:18:51.120855 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c3a202-44cd-46db-b92a-135992ebab33 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:51.123545 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-55c3a202-44cd-46db-b92a-135992ebab33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:18:51.149305 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c3a202-44cd-46db-b92a-135992ebab33 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:51.158621 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206f7d86-08f2-4314-9067-8a1e14b01ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:51.159470 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206f7d86-08f2-4314-9067-8a1e14b01ded admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:51.159761 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206f7d86-08f2-4314-9067-8a1e14b01ded admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:51.160032 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206f7d86-08f2-4314-9067-8a1e14b01ded admin admin] PATH_INFO: `/v3/users/25be51a68ccf4ebbbaac8b4fd0633bc3` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:51.163597 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206f7d86-08f2-4314-9067-8a1e14b01ded admin admin] RBAC: Authorizing `identity:delete_user(user_id=25be51a68ccf4ebbbaac8b4fd0633bc3)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:51.167993 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206f7d86-08f2-4314-9067-8a1e14b01ded admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:51.198038 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca77afe-6789-433a-a418-37456dc27b6d admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:51.203708 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d662dd91-6741-484a-b5df-6803ed8059cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:51.204339 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d662dd91-6741-484a-b5df-6803ed8059cc service neutron] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:51.204622 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d662dd91-6741-484a-b5df-6803ed8059cc service neutron] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:51.205505 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d662dd91-6741-484a-b5df-6803ed8059cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:51.217181 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d662dd91-6741-484a-b5df-6803ed8059cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:51.219996 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d662dd91-6741-484a-b5df-6803ed8059cc service neutron] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:51.230673 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 819/1646] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:18:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 25 03:18:51.506829 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-206f7d86-08f2-4314-9067-8a1e14b01ded admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25be51a68ccf4ebbbaac8b4fd0633bc3'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:18:51.629507 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-206f7d86-08f2-4314-9067-8a1e14b01ded admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25be51a68ccf4ebbbaac8b4fd0633bc3'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:18:51.639635 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-206f7d86-08f2-4314-9067-8a1e14b01ded admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25be51a68ccf4ebbbaac8b4fd0633bc3'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:18:51.659601 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-206f7d86-08f2-4314-9067-8a1e14b01ded admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25be51a68ccf4ebbbaac8b4fd0633bc3'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:18:51.703264 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-206f7d86-08f2-4314-9067-8a1e14b01ded admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25be51a68ccf4ebbbaac8b4fd0633bc3'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:18:51.719865 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 828/1647] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:18:51 2022] DELETE /identity/v3/users/25be51a68ccf4ebbbaac8b4fd0633bc3 => generated 0 bytes in 600 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:18:51.724670 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80f24f1-7e9d-4eec-ab60-0d00342eb454 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:51.724963 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80f24f1-7e9d-4eec-ab60-0d00342eb454 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:51.725290 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80f24f1-7e9d-4eec-ab60-0d00342eb454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:51.772146 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-f80f24f1-7e9d-4eec-ab60-0d00342eb454 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:18:51.773418 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-f80f24f1-7e9d-4eec-ab60-0d00342eb454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:18:51.954252 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:18:51.980151 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 820/1648] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:18:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:18:51.997041 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206f7d86-08f2-4314-9067-8a1e14b01ded admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:51.998900 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-206f7d86-08f2-4314-9067-8a1e14b01ded admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:18:52.089511 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206f7d86-08f2-4314-9067-8a1e14b01ded admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:52.097073 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d29be3c-8bf0-442e-96e5-aa09b034316e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:52.097712 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d29be3c-8bf0-442e-96e5-aa09b034316e service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:52.097994 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d29be3c-8bf0-442e-96e5-aa09b034316e service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:52.098270 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d29be3c-8bf0-442e-96e5-aa09b034316e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:52.117093 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d29be3c-8bf0-442e-96e5-aa09b034316e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:52.119490 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d29be3c-8bf0-442e-96e5-aa09b034316e service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:52.124491 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:18:52.135327 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 829/1649] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:18:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:18:52.735725 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80f24f1-7e9d-4eec-ab60-0d00342eb454 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:52.736795 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-f80f24f1-7e9d-4eec-ab60-0d00342eb454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:18:52.760362 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80f24f1-7e9d-4eec-ab60-0d00342eb454 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:52.778095 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779f3f30-b9d2-42cf-885f-e325e0228175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:52.779502 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779f3f30-b9d2-42cf-885f-e325e0228175 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:52.779927 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779f3f30-b9d2-42cf-885f-e325e0228175 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:52.780832 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779f3f30-b9d2-42cf-885f-e325e0228175 admin admin] PATH_INFO: `/v3/projects/e1082e3330ff43b58852314736751cc6` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:52.793166 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779f3f30-b9d2-42cf-885f-e325e0228175 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1082e3330ff43b58852314736751cc6)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:52.793166 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779f3f30-b9d2-42cf-885f-e325e0228175 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:53.049189 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-779f3f30-b9d2-42cf-885f-e325e0228175 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1082e3330ff43b58852314736751cc6', 'request_id': 'req-779f3f30-b9d2-42cf-885f-e325e0228175'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:18:53.125946 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-779f3f30-b9d2-42cf-885f-e325e0228175 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1082e3330ff43b58852314736751cc6', 'request_id': 'req-779f3f30-b9d2-42cf-885f-e325e0228175'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:18:53.138890 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-779f3f30-b9d2-42cf-885f-e325e0228175 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1082e3330ff43b58852314736751cc6', 'request_id': 'req-779f3f30-b9d2-42cf-885f-e325e0228175'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:18:53.173223 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-779f3f30-b9d2-42cf-885f-e325e0228175 admin admin] The token cache is being invalidate because project e1082e3330ff43b58852314736751cc6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81995) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:18:53.173526 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-779f3f30-b9d2-42cf-885f-e325e0228175 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:18:53.192032 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-779f3f30-b9d2-42cf-885f-e325e0228175 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:18:53.208589 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 821/1650] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:18:52 2022] DELETE /identity/v3/projects/e1082e3330ff43b58852314736751cc6 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:18:53.210591 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff36221e-2b9a-4200-a559-a94657d1dd2e None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:53.211134 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff36221e-2b9a-4200-a559-a94657d1dd2e None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:53.211134 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff36221e-2b9a-4200-a559-a94657d1dd2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:53.284278 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-ff36221e-2b9a-4200-a559-a94657d1dd2e None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:18:53.285547 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-ff36221e-2b9a-4200-a559-a94657d1dd2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:18:53.476793 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 830/1651] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:18:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:18:53.485991 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45155c61-e0b9-4416-93a2-91a66206736f None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:53.486396 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45155c61-e0b9-4416-93a2-91a66206736f None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:53.486751 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45155c61-e0b9-4416-93a2-91a66206736f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:53.571644 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-45155c61-e0b9-4416-93a2-91a66206736f None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:18:53.573916 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-45155c61-e0b9-4416-93a2-91a66206736f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:18:53.589498 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 822/1652] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:18:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:18:53.594329 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff36221e-2b9a-4200-a559-a94657d1dd2e None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:53.619032 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff36221e-2b9a-4200-a559-a94657d1dd2e None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:53.628273 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79707cf-327b-4608-ab29-4c483dc430f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:53.628968 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79707cf-327b-4608-ab29-4c483dc430f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:53.629330 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79707cf-327b-4608-ab29-4c483dc430f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:53.629667 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79707cf-327b-4608-ab29-4c483dc430f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:53.634723 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79707cf-327b-4608-ab29-4c483dc430f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:18:53.635118 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79707cf-327b-4608-ab29-4c483dc430f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:53.640376 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79707cf-327b-4608-ab29-4c483dc430f6 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:53.687643 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 831/1653] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Aug 25 03:18:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:18:53.691007 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45155c61-e0b9-4416-93a2-91a66206736f None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:53.709424 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45155c61-e0b9-4416-93a2-91a66206736f None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:53.728512 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872a3c8d-56bb-490b-8f68-2cf17bae508c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:53.730478 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872a3c8d-56bb-490b-8f68-2cf17bae508c admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:53.732113 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872a3c8d-56bb-490b-8f68-2cf17bae508c admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:53.732113 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872a3c8d-56bb-490b-8f68-2cf17bae508c admin admin] PATH_INFO: `/v3/projects` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:53.732883 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872a3c8d-56bb-490b-8f68-2cf17bae508c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:53.735224 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872a3c8d-56bb-490b-8f68-2cf17bae508c admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:53.856988 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 823/1654] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:18:53 2022] POST /identity/v3/projects => generated 405 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:18:53.858316 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79707cf-327b-4608-ab29-4c483dc430f6 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:53.880552 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79707cf-327b-4608-ab29-4c483dc430f6 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:54.084588 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67234c09-b15f-461d-ad64-4ae52fa47875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:54.084949 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67234c09-b15f-461d-ad64-4ae52fa47875 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:54.085488 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67234c09-b15f-461d-ad64-4ae52fa47875 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:54.085973 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67234c09-b15f-461d-ad64-4ae52fa47875 admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:54.087089 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67234c09-b15f-461d-ad64-4ae52fa47875 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:54.093381 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67234c09-b15f-461d-ad64-4ae52fa47875 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:54.223900 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 832/1655] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:18:53 2022] POST /identity/v3/users => generated 303 bytes in 362 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:18:54.232906 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872a3c8d-56bb-490b-8f68-2cf17bae508c admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:54.246694 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872a3c8d-56bb-490b-8f68-2cf17bae508c admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:54.268542 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f162a8f9-4ca2-4d6a-bbb3-2533365f0dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:54.268646 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f162a8f9-4ca2-4d6a-bbb3-2533365f0dc0 admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:54.269016 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f162a8f9-4ca2-4d6a-bbb3-2533365f0dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:54.270481 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f162a8f9-4ca2-4d6a-bbb3-2533365f0dc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:54.273572 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f162a8f9-4ca2-4d6a-bbb3-2533365f0dc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:54.285117 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f162a8f9-4ca2-4d6a-bbb3-2533365f0dc0 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:54.320729 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 824/1656] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:18:54 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:18:54.325554 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67234c09-b15f-461d-ad64-4ae52fa47875 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:54.341222 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67234c09-b15f-461d-ad64-4ae52fa47875 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:54.354812 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beb4bc2-7d66-44a3-a447-e33afa7ffe26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:54.355493 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beb4bc2-7d66-44a3-a447-e33afa7ffe26 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:54.355713 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beb4bc2-7d66-44a3-a447-e33afa7ffe26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:54.355992 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beb4bc2-7d66-44a3-a447-e33afa7ffe26 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:54.357215 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1beb4bc2-7d66-44a3-a447-e33afa7ffe26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:54.362679 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1beb4bc2-7d66-44a3-a447-e33afa7ffe26 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:54.401197 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 833/1657] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:18:54 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:18:54.405905 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f162a8f9-4ca2-4d6a-bbb3-2533365f0dc0 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:54.426013 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f162a8f9-4ca2-4d6a-bbb3-2533365f0dc0 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:54.438950 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de67dc2f-7afe-4902-b2f4-da273566eb59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:54.439757 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de67dc2f-7afe-4902-b2f4-da273566eb59 admin admin] REQUEST_METHOD: `PUT` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:54.440167 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de67dc2f-7afe-4902-b2f4-da273566eb59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:54.440531 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de67dc2f-7afe-4902-b2f4-da273566eb59 admin admin] PATH_INFO: `/v3/projects/ad5f85a315d6404fa21d9bf8aa985c9b/users/49405a748ab54056ba8ba9f996381135/roles/3255f77192524a16bc405173106871fc` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:54.483429 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de67dc2f-7afe-4902-b2f4-da273566eb59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad5f85a315d6404fa21d9bf8aa985c9b, user_id=49405a748ab54056ba8ba9f996381135, role_id=3255f77192524a16bc405173106871fc)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:54.486236 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de67dc2f-7afe-4902-b2f4-da273566eb59 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:54.572523 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 825/1658] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:18:54 2022] PUT /identity/v3/projects/ad5f85a315d6404fa21d9bf8aa985c9b/users/49405a748ab54056ba8ba9f996381135/roles/3255f77192524a16bc405173106871fc => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:18:54.583026 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e63179-f902-4ca8-a80e-6970b19428b5 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:54.583316 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e63179-f902-4ca8-a80e-6970b19428b5 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:54.583600 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e63179-f902-4ca8-a80e-6970b19428b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:54.626655 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-42e63179-f902-4ca8-a80e-6970b19428b5 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:18:54.627527 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-42e63179-f902-4ca8-a80e-6970b19428b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:18:54.725470 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 834/1659] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:18:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:18:54.758060 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de67dc2f-7afe-4902-b2f4-da273566eb59 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:54.761180 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-de67dc2f-7afe-4902-b2f4-da273566eb59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:18:54.948762 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de67dc2f-7afe-4902-b2f4-da273566eb59 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:54.957447 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e67cb34-101d-4532-828c-7b6eaf19b4c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:54.958359 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e67cb34-101d-4532-828c-7b6eaf19b4c8 service neutron] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:54.959701 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e67cb34-101d-4532-828c-7b6eaf19b4c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:54.960157 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e67cb34-101d-4532-828c-7b6eaf19b4c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:54.976092 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e67cb34-101d-4532-828c-7b6eaf19b4c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:54.978199 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e67cb34-101d-4532-828c-7b6eaf19b4c8 service neutron] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:54.994703 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 826/1660] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:18:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:18:55.159275 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e63179-f902-4ca8-a80e-6970b19428b5 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:55.165327 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-42e63179-f902-4ca8-a80e-6970b19428b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:18:55.177218 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:18:55.202592 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e63179-f902-4ca8-a80e-6970b19428b5 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:55.216941 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0836852-112e-4077-b910-91cd2b6c25a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:55.216941 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0836852-112e-4077-b910-91cd2b6c25a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:55.217130 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0836852-112e-4077-b910-91cd2b6c25a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:55.217256 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0836852-112e-4077-b910-91cd2b6c25a7 admin admin] PATH_INFO: `/v3/projects/ffe88d801c9643c2a6cf9fce35080ad1` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:55.219033 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0836852-112e-4077-b910-91cd2b6c25a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffe88d801c9643c2a6cf9fce35080ad1)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:55.220084 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: WARNING py.warnings [None req-b0836852-112e-4077-b910-91cd2b6c25a7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 25 03:18:55.220084 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: warnings.warn(msg) Aug 25 03:18:55.220084 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: Aug 25 03:18:55.229350 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0836852-112e-4077-b910-91cd2b6c25a7 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:55.502403 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-b0836852-112e-4077-b910-91cd2b6c25a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffe88d801c9643c2a6cf9fce35080ad1', 'request_id': 'req-b0836852-112e-4077-b910-91cd2b6c25a7'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:18:55.658195 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-b0836852-112e-4077-b910-91cd2b6c25a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffe88d801c9643c2a6cf9fce35080ad1', 'request_id': 'req-b0836852-112e-4077-b910-91cd2b6c25a7'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:18:55.665585 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-b0836852-112e-4077-b910-91cd2b6c25a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffe88d801c9643c2a6cf9fce35080ad1', 'request_id': 'req-b0836852-112e-4077-b910-91cd2b6c25a7'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:18:55.706802 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-b0836852-112e-4077-b910-91cd2b6c25a7 admin admin] The token cache is being invalidate because project ffe88d801c9643c2a6cf9fce35080ad1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:18:55.707499 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-b0836852-112e-4077-b910-91cd2b6c25a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:18:55.708305 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-b0836852-112e-4077-b910-91cd2b6c25a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:18:55.710075 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 835/1661] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:18:55 2022] DELETE /identity/v3/projects/ffe88d801c9643c2a6cf9fce35080ad1 => generated 0 bytes in 551 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:18:55.714265 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e67cb34-101d-4532-828c-7b6eaf19b4c8 service neutron] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:55.728557 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e67cb34-101d-4532-828c-7b6eaf19b4c8 service neutron] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:55.783816 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:18:55.884870 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fec70b-927d-460b-ab50-fe7640ea7391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:55.885763 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fec70b-927d-460b-ab50-fe7640ea7391 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:55.889616 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fec70b-927d-460b-ab50-fe7640ea7391 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:55.889616 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fec70b-927d-460b-ab50-fe7640ea7391 admin admin] PATH_INFO: `/v3/users/b5cbd421afee4e08928dacc23e119736` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:55.889616 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fec70b-927d-460b-ab50-fe7640ea7391 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5cbd421afee4e08928dacc23e119736)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:55.892141 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fec70b-927d-460b-ab50-fe7640ea7391 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:56.185421 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-29fec70b-927d-460b-ab50-fe7640ea7391 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5cbd421afee4e08928dacc23e119736'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:18:56.330827 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-29fec70b-927d-460b-ab50-fe7640ea7391 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5cbd421afee4e08928dacc23e119736'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:18:56.334020 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-29fec70b-927d-460b-ab50-fe7640ea7391 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5cbd421afee4e08928dacc23e119736'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:18:56.334880 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-29fec70b-927d-460b-ab50-fe7640ea7391 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5cbd421afee4e08928dacc23e119736'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:18:56.373923 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-29fec70b-927d-460b-ab50-fe7640ea7391 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5cbd421afee4e08928dacc23e119736'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:18:56.423368 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 827/1662] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:18:55 2022] DELETE /identity/v3/users/b5cbd421afee4e08928dacc23e119736 => generated 0 bytes in 709 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:18:56.528906 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0836852-112e-4077-b910-91cd2b6c25a7 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:56.530978 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-b0836852-112e-4077-b910-91cd2b6c25a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:18:56.555298 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0836852-112e-4077-b910-91cd2b6c25a7 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:56.715771 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0f021d-3e23-4b7b-afe5-6c0ea77c1860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:56.716790 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0f021d-3e23-4b7b-afe5-6c0ea77c1860 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:56.717274 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0f021d-3e23-4b7b-afe5-6c0ea77c1860 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:56.717699 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0f021d-3e23-4b7b-afe5-6c0ea77c1860 admin admin] PATH_INFO: `/v3/users/30737ce2040a4b5aac1c6c587abb9a81` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:56.719450 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0f021d-3e23-4b7b-afe5-6c0ea77c1860 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30737ce2040a4b5aac1c6c587abb9a81)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:56.726463 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0f021d-3e23-4b7b-afe5-6c0ea77c1860 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:56.928626 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-ef0f021d-3e23-4b7b-afe5-6c0ea77c1860 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30737ce2040a4b5aac1c6c587abb9a81'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:18:57.034209 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-ef0f021d-3e23-4b7b-afe5-6c0ea77c1860 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30737ce2040a4b5aac1c6c587abb9a81'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:18:57.034861 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-ef0f021d-3e23-4b7b-afe5-6c0ea77c1860 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30737ce2040a4b5aac1c6c587abb9a81'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:18:57.043856 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-ef0f021d-3e23-4b7b-afe5-6c0ea77c1860 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30737ce2040a4b5aac1c6c587abb9a81'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:18:57.076610 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-ef0f021d-3e23-4b7b-afe5-6c0ea77c1860 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30737ce2040a4b5aac1c6c587abb9a81'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:18:57.094938 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 836/1663] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:18:56 2022] DELETE /identity/v3/users/30737ce2040a4b5aac1c6c587abb9a81 => generated 0 bytes in 567 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:18:57.100343 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5dc1b0-f672-4eb3-8ad4-fc7f628cd767 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:57.100632 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5dc1b0-f672-4eb3-8ad4-fc7f628cd767 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:57.100908 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5dc1b0-f672-4eb3-8ad4-fc7f628cd767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:57.147414 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-7e5dc1b0-f672-4eb3-8ad4-fc7f628cd767 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:18:57.147414 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-7e5dc1b0-f672-4eb3-8ad4-fc7f628cd767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:18:57.269179 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 828/1664] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:18:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:18:57.289786 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0f021d-3e23-4b7b-afe5-6c0ea77c1860 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:57.291900 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-ef0f021d-3e23-4b7b-afe5-6c0ea77c1860 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:18:57.322365 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:18:57.447871 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0f021d-3e23-4b7b-afe5-6c0ea77c1860 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:57.454180 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2aabd8-7e61-4fd6-ac09-92a803d57951 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:57.455005 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2aabd8-7e61-4fd6-ac09-92a803d57951 service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:57.455582 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2aabd8-7e61-4fd6-ac09-92a803d57951 service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:57.456054 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2aabd8-7e61-4fd6-ac09-92a803d57951 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:57.486229 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2aabd8-7e61-4fd6-ac09-92a803d57951 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:57.490453 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2aabd8-7e61-4fd6-ac09-92a803d57951 service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:57.520032 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 837/1665] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:18:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:18:57.657748 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b375c27e-d2fe-4dd4-820e-adfad7265854 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:57.658562 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b375c27e-d2fe-4dd4-820e-adfad7265854 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:57.661050 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b375c27e-d2fe-4dd4-820e-adfad7265854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:57.736003 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-b375c27e-d2fe-4dd4-820e-adfad7265854 None None] MFA Rules not processed for user `827bfe2d861146b3aa67cfe147dc16d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:18:57.739316 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-b375c27e-d2fe-4dd4-820e-adfad7265854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:18:57.815793 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:18:57.926397 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 829/1666] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:18:57 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:18:57.936591 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3b8c38-7081-4865-a64e-c2540e751734 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:57.937101 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3b8c38-7081-4865-a64e-c2540e751734 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:57.937180 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3b8c38-7081-4865-a64e-c2540e751734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:57.980058 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-2f3b8c38-7081-4865-a64e-c2540e751734 None None] MFA Rules not processed for user `827bfe2d861146b3aa67cfe147dc16d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:18:57.988809 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-2f3b8c38-7081-4865-a64e-c2540e751734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:18:58.003838 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 838/1667] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:18:57 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:18:58.019459 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b375c27e-d2fe-4dd4-820e-adfad7265854 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:58.023162 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-b375c27e-d2fe-4dd4-820e-adfad7265854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:18:58.135410 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3b8c38-7081-4865-a64e-c2540e751734 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:58.140453 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-2f3b8c38-7081-4865-a64e-c2540e751734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:18:58.185824 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3b8c38-7081-4865-a64e-c2540e751734 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:58.205186 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f51deb7-87bb-4201-830e-f4346409abe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:58.208308 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f51deb7-87bb-4201-830e-f4346409abe8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:58.210314 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f51deb7-87bb-4201-830e-f4346409abe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:58.210314 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f51deb7-87bb-4201-830e-f4346409abe8 admin admin] PATH_INFO: `/v3/projects/42d84da7a4154e4c8075f60b563cbfe5` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:58.216732 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f51deb7-87bb-4201-830e-f4346409abe8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42d84da7a4154e4c8075f60b563cbfe5)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:58.221470 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f51deb7-87bb-4201-830e-f4346409abe8 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:58.330763 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b375c27e-d2fe-4dd4-820e-adfad7265854 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:58.352934 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ce981f-47e1-484d-b18b-60f6132a932d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cd171bf4df94f8fab397e42835feeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:58.353964 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ce981f-47e1-484d-b18b-60f6132a932d service nova] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:58.354477 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ce981f-47e1-484d-b18b-60f6132a932d service nova] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:58.357438 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ce981f-47e1-484d-b18b-60f6132a932d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:58.379710 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ce981f-47e1-484d-b18b-60f6132a932d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:58.389765 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ce981f-47e1-484d-b18b-60f6132a932d service nova] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:58.403617 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 830/1668] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:18:58 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:18:58.405208 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a86890-5057-43e2-af63-881d3edbc132 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:58.409689 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a86890-5057-43e2-af63-881d3edbc132 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:58.410318 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a86890-5057-43e2-af63-881d3edbc132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:58.476002 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-0f51deb7-87bb-4201-830e-f4346409abe8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42d84da7a4154e4c8075f60b563cbfe5', 'request_id': 'req-0f51deb7-87bb-4201-830e-f4346409abe8'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:18:58.494714 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-15a86890-5057-43e2-af63-881d3edbc132 None None] MFA Rules not processed for user `49405a748ab54056ba8ba9f996381135`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:18:58.500243 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-15a86890-5057-43e2-af63-881d3edbc132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:18:58.554658 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-0f51deb7-87bb-4201-830e-f4346409abe8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42d84da7a4154e4c8075f60b563cbfe5', 'request_id': 'req-0f51deb7-87bb-4201-830e-f4346409abe8'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:18:58.564608 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-0f51deb7-87bb-4201-830e-f4346409abe8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42d84da7a4154e4c8075f60b563cbfe5', 'request_id': 'req-0f51deb7-87bb-4201-830e-f4346409abe8'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:18:58.594702 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-0f51deb7-87bb-4201-830e-f4346409abe8 admin admin] The token cache is being invalidate because project 42d84da7a4154e4c8075f60b563cbfe5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:18:58.595081 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-0f51deb7-87bb-4201-830e-f4346409abe8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:18:58.595971 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-0f51deb7-87bb-4201-830e-f4346409abe8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:18:58.597935 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 839/1669] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:18:58 2022] DELETE /identity/v3/projects/42d84da7a4154e4c8075f60b563cbfe5 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:18:58.602704 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed37637-40f5-4487-92c1-88bdd9c61cb7 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:58.603557 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed37637-40f5-4487-92c1-88bdd9c61cb7 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:58.603557 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed37637-40f5-4487-92c1-88bdd9c61cb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:58.651125 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-fed37637-40f5-4487-92c1-88bdd9c61cb7 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:18:58.661413 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-fed37637-40f5-4487-92c1-88bdd9c61cb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:18:58.661540 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 831/1670] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:18:58 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:18:58.665089 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a86890-5057-43e2-af63-881d3edbc132 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:58.666769 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-15a86890-5057-43e2-af63-881d3edbc132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:18:58.796308 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 840/1671] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:18:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:18:58.797986 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed37637-40f5-4487-92c1-88bdd9c61cb7 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:58.799513 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-fed37637-40f5-4487-92c1-88bdd9c61cb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:18:58.809154 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a86890-5057-43e2-af63-881d3edbc132 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:58.816556 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b177df-d2a8-4271-830b-f58986518544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:58.819168 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b177df-d2a8-4271-830b-f58986518544 admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:58.819806 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b177df-d2a8-4271-830b-f58986518544 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:58.820307 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b177df-d2a8-4271-830b-f58986518544 admin admin] PATH_INFO: `/v3/projects` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:58.821418 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b177df-d2a8-4271-830b-f58986518544 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:58.825805 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b177df-d2a8-4271-830b-f58986518544 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:58.827143 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed37637-40f5-4487-92c1-88bdd9c61cb7 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:58.839024 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430b221e-b998-44bf-a0f0-eda9a0af908f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:58.839698 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430b221e-b998-44bf-a0f0-eda9a0af908f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:58.840083 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430b221e-b998-44bf-a0f0-eda9a0af908f admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:58.840362 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430b221e-b998-44bf-a0f0-eda9a0af908f admin admin] PATH_INFO: `/v3/projects/fe809c1bb51c4f8c9380c1ada3643ce9` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:58.841967 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430b221e-b998-44bf-a0f0-eda9a0af908f admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe809c1bb51c4f8c9380c1ada3643ce9)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:58.844403 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430b221e-b998-44bf-a0f0-eda9a0af908f admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:58.876845 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:18:58.988092 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 832/1672] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:18:58 2022] POST /identity/v3/projects => generated 403 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:18:58.989025 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37abb270-d3b4-44d6-8adb-742e95e791dd None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:58.989317 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37abb270-d3b4-44d6-8adb-742e95e791dd None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:58.990451 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37abb270-d3b4-44d6-8adb-742e95e791dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:59.023707 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-37abb270-d3b4-44d6-8adb-742e95e791dd None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:18:59.032821 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-37abb270-d3b4-44d6-8adb-742e95e791dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:18:59.098143 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-430b221e-b998-44bf-a0f0-eda9a0af908f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe809c1bb51c4f8c9380c1ada3643ce9', 'request_id': 'req-430b221e-b998-44bf-a0f0-eda9a0af908f'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:18:59.189538 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 833/1673] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:18:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:18:59.190702 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37abb270-d3b4-44d6-8adb-742e95e791dd None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:59.201420 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-430b221e-b998-44bf-a0f0-eda9a0af908f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe809c1bb51c4f8c9380c1ada3643ce9', 'request_id': 'req-430b221e-b998-44bf-a0f0-eda9a0af908f'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:18:59.222349 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-430b221e-b998-44bf-a0f0-eda9a0af908f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe809c1bb51c4f8c9380c1ada3643ce9', 'request_id': 'req-430b221e-b998-44bf-a0f0-eda9a0af908f'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:18:59.235779 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37abb270-d3b4-44d6-8adb-742e95e791dd None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:59.255815 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-430b221e-b998-44bf-a0f0-eda9a0af908f admin admin] The token cache is being invalidate because project fe809c1bb51c4f8c9380c1ada3643ce9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:18:59.256380 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-430b221e-b998-44bf-a0f0-eda9a0af908f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:18:59.258706 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-430b221e-b998-44bf-a0f0-eda9a0af908f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:18:59.267961 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 841/1674] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:18:58 2022] DELETE /identity/v3/projects/fe809c1bb51c4f8c9380c1ada3643ce9 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:18:59.274851 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430b221e-b998-44bf-a0f0-eda9a0af908f admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:59.279252 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-430b221e-b998-44bf-a0f0-eda9a0af908f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:18:59.397707 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22771dd0-3077-48a8-a434-497ff0687e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:59.399081 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22771dd0-3077-48a8-a434-497ff0687e60 admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:59.405675 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22771dd0-3077-48a8-a434-497ff0687e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:59.408118 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22771dd0-3077-48a8-a434-497ff0687e60 admin admin] PATH_INFO: `/v3/users` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:59.410897 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22771dd0-3077-48a8-a434-497ff0687e60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:59.415720 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22771dd0-3077-48a8-a434-497ff0687e60 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:59.425482 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430b221e-b998-44bf-a0f0-eda9a0af908f admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:59.434900 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3523b12-ee0c-4958-9eb1-d98ae11e063c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:59.434900 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3523b12-ee0c-4958-9eb1-d98ae11e063c admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:59.434900 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3523b12-ee0c-4958-9eb1-d98ae11e063c admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:59.434900 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3523b12-ee0c-4958-9eb1-d98ae11e063c admin admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:59.435221 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3523b12-ee0c-4958-9eb1-d98ae11e063c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:18:59.435221 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3523b12-ee0c-4958-9eb1-d98ae11e063c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:59.435831 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3523b12-ee0c-4958-9eb1-d98ae11e063c admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:59.450497 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 842/1675] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Aug 25 03:18:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:18:59.457806 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3523b12-ee0c-4958-9eb1-d98ae11e063c admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:59.464718 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-d3523b12-ee0c-4958-9eb1-d98ae11e063c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:18:59.490413 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:18:59.509174 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3523b12-ee0c-4958-9eb1-d98ae11e063c admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:59.519269 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc49b479-b14a-4520-9b4e-7d71412986ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:59.519935 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc49b479-b14a-4520-9b4e-7d71412986ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:59.520220 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc49b479-b14a-4520-9b4e-7d71412986ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:59.520499 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc49b479-b14a-4520-9b4e-7d71412986ba admin admin] PATH_INFO: `/v3/users/1a186753581e40f3a51356637f0bd358` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:59.521650 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc49b479-b14a-4520-9b4e-7d71412986ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a186753581e40f3a51356637f0bd358)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:59.524056 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc49b479-b14a-4520-9b4e-7d71412986ba admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:59.610213 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 834/1676] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:18:59 2022] POST /identity/v3/users => generated 308 bytes in 420 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:18:59.611150 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22771dd0-3077-48a8-a434-497ff0687e60 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:59.648958 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22771dd0-3077-48a8-a434-497ff0687e60 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:59.655816 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dcd6d8-b058-43d4-b149-704937e18b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:59.656888 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dcd6d8-b058-43d4-b149-704937e18b8c admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:59.661847 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dcd6d8-b058-43d4-b149-704937e18b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:59.661847 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dcd6d8-b058-43d4-b149-704937e18b8c admin admin] PATH_INFO: `/v3/projects` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:59.661847 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dcd6d8-b058-43d4-b149-704937e18b8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:59.661847 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dcd6d8-b058-43d4-b149-704937e18b8c admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:59.800901 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 835/1677] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:18:59 2022] POST /identity/v3/projects => generated 409 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:18:59.802845 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dcd6d8-b058-43d4-b149-704937e18b8c admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:59.802970 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-cc49b479-b14a-4520-9b4e-7d71412986ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a186753581e40f3a51356637f0bd358'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:18:59.805031 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-91dcd6d8-b058-43d4-b149-704937e18b8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:18:59.907640 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-cc49b479-b14a-4520-9b4e-7d71412986ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a186753581e40f3a51356637f0bd358'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:18:59.913014 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-cc49b479-b14a-4520-9b4e-7d71412986ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a186753581e40f3a51356637f0bd358'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:18:59.924916 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dcd6d8-b058-43d4-b149-704937e18b8c admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:59.927894 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-cc49b479-b14a-4520-9b4e-7d71412986ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a186753581e40f3a51356637f0bd358'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:18:59.935420 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102575c4-9190-4e03-ae5d-ee1f535b1899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:59.950376 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102575c4-9190-4e03-ae5d-ee1f535b1899 admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:59.951753 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102575c4-9190-4e03-ae5d-ee1f535b1899 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:59.952762 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102575c4-9190-4e03-ae5d-ee1f535b1899 admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:59.957159 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102575c4-9190-4e03-ae5d-ee1f535b1899 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:59.960308 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102575c4-9190-4e03-ae5d-ee1f535b1899 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:59.965516 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-cc49b479-b14a-4520-9b4e-7d71412986ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a186753581e40f3a51356637f0bd358'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:18:59.982806 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 843/1678] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:18:59 2022] DELETE /identity/v3/users/1a186753581e40f3a51356637f0bd358 => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:18:59.984233 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860c8424-223f-412b-baef-97904e99f580 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:59.984526 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860c8424-223f-412b-baef-97904e99f580 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:59.984804 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860c8424-223f-412b-baef-97904e99f580 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:59.989429 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 844/1679] 10.222.0.58 () {54 vars in 912 bytes} [Thu Aug 25 03:18:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:18:59.992132 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860c8424-223f-412b-baef-97904e99f580 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:59.993715 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 836/1680] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:18:59 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:18:59.994968 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102575c4-9190-4e03-ae5d-ee1f535b1899 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:59.998730 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-102575c4-9190-4e03-ae5d-ee1f535b1899 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:00.004144 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-860c8424-223f-412b-baef-97904e99f580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:00.017609 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102575c4-9190-4e03-ae5d-ee1f535b1899 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:00.026668 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:19:00.134543 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3996f7c5-a549-43b0-857b-ee91d803d260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:00.137105 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3996f7c5-a549-43b0-857b-ee91d803d260 admin admin] REQUEST_METHOD: `PUT` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:00.137945 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3996f7c5-a549-43b0-857b-ee91d803d260 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:00.138357 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3996f7c5-a549-43b0-857b-ee91d803d260 admin admin] PATH_INFO: `/v3/projects/b31b2419e7d84fd68089a257a1baf479/users/2e2cb997c7f047328ec6e7837f67d29f/roles/62f19ea272b447ccaab1576cd9bf4fd9` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:00.172799 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860c8424-223f-412b-baef-97904e99f580 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:00.182552 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3996f7c5-a549-43b0-857b-ee91d803d260 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b31b2419e7d84fd68089a257a1baf479, user_id=2e2cb997c7f047328ec6e7837f67d29f, role_id=62f19ea272b447ccaab1576cd9bf4fd9)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:00.185801 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3996f7c5-a549-43b0-857b-ee91d803d260 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:00.190889 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba24595-284a-447c-9088-5ad2a05891c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:00.192242 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba24595-284a-447c-9088-5ad2a05891c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:00.193116 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba24595-284a-447c-9088-5ad2a05891c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:00.193116 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba24595-284a-447c-9088-5ad2a05891c1 admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:00.195947 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba24595-284a-447c-9088-5ad2a05891c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:00.199558 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba24595-284a-447c-9088-5ad2a05891c1 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:00.280711 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 837/1681] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:18:59 2022] PUT /identity/v3/projects/b31b2419e7d84fd68089a257a1baf479/users/2e2cb997c7f047328ec6e7837f67d29f/roles/62f19ea272b447ccaab1576cd9bf4fd9 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:19:00.282769 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5056d0-bdeb-48e8-b9f7-0642ca2d571f None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:00.283875 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5056d0-bdeb-48e8-b9f7-0642ca2d571f None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:00.284381 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5056d0-bdeb-48e8-b9f7-0642ca2d571f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:00.371844 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-9f5056d0-bdeb-48e8-b9f7-0642ca2d571f None None] MFA Rules not processed for user `a0a1a7f091774842907fd08de15e8058`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:19:00.375174 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-9f5056d0-bdeb-48e8-b9f7-0642ca2d571f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:00.385748 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 845/1682] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:18:59 2022] POST /identity/v3/users => generated 305 bytes in 396 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:19:00.386782 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba24595-284a-447c-9088-5ad2a05891c1 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:00.429672 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba24595-284a-447c-9088-5ad2a05891c1 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:00.549318 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 838/1683] 10.222.0.58 () {58 vars in 1009 bytes} [Thu Aug 25 03:19:00 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:19:00.554124 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5056d0-bdeb-48e8-b9f7-0642ca2d571f None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:00.685137 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c57e32-1f6d-41be-9976-7d75e4949c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:00.685863 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c57e32-1f6d-41be-9976-7d75e4949c63 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:00.686139 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c57e32-1f6d-41be-9976-7d75e4949c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:00.686475 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c57e32-1f6d-41be-9976-7d75e4949c63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:00.687382 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c57e32-1f6d-41be-9976-7d75e4949c63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:00.689342 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c57e32-1f6d-41be-9976-7d75e4949c63 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:00.719523 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 846/1684] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:19:00 2022] GET /identity/v3/roles => generated 1420 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:19:00.720625 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c57e32-1f6d-41be-9976-7d75e4949c63 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:00.722790 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-83c57e32-1f6d-41be-9976-7d75e4949c63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:00.740089 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c57e32-1f6d-41be-9976-7d75e4949c63 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:00.744428 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0623be18-02c5-4902-9fe5-7c0f18d6ce35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:00.745089 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0623be18-02c5-4902-9fe5-7c0f18d6ce35 service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:00.745370 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0623be18-02c5-4902-9fe5-7c0f18d6ce35 service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:00.745641 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0623be18-02c5-4902-9fe5-7c0f18d6ce35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:00.774321 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0623be18-02c5-4902-9fe5-7c0f18d6ce35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:00.779996 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0623be18-02c5-4902-9fe5-7c0f18d6ce35 service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:00.781984 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5056d0-bdeb-48e8-b9f7-0642ca2d571f None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:00.807268 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 847/1685] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:19:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:19:00.807514 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d872dfa5-e7d5-43b1-84a1-8ad74c811245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:00.808643 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0623be18-02c5-4902-9fe5-7c0f18d6ce35 service neutron] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:00.819329 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d872dfa5-e7d5-43b1-84a1-8ad74c811245 admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:00.825116 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d872dfa5-e7d5-43b1-84a1-8ad74c811245 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:00.825116 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d872dfa5-e7d5-43b1-84a1-8ad74c811245 admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:00.825116 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d872dfa5-e7d5-43b1-84a1-8ad74c811245 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:00.829402 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d872dfa5-e7d5-43b1-84a1-8ad74c811245 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:00.837611 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0623be18-02c5-4902-9fe5-7c0f18d6ce35 service neutron] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:00.880947 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f5ff81-c52a-4d07-b296-5b1ceffb61ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:00.887766 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f5ff81-c52a-4d07-b296-5b1ceffb61ff admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:00.888225 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 839/1686] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:19:00 2022] GET /identity/v3/roles => generated 1420 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:19:00.888550 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f5ff81-c52a-4d07-b296-5b1ceffb61ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:00.889036 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f5ff81-c52a-4d07-b296-5b1ceffb61ff admin admin] PATH_INFO: `/v3/projects/b31b2419e7d84fd68089a257a1baf479/users/2e2cb997c7f047328ec6e7837f67d29f/roles/62f19ea272b447ccaab1576cd9bf4fd9` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:00.890959 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ef29d7-7f5f-4c87-8f9b-52261a0d8d9e None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:00.891478 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ef29d7-7f5f-4c87-8f9b-52261a0d8d9e None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:00.891589 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ef29d7-7f5f-4c87-8f9b-52261a0d8d9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:00.912412 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:19:00.919924 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f5ff81-c52a-4d07-b296-5b1ceffb61ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=b31b2419e7d84fd68089a257a1baf479, user_id=2e2cb997c7f047328ec6e7837f67d29f, role_id=62f19ea272b447ccaab1576cd9bf4fd9)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:00.928375 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f5ff81-c52a-4d07-b296-5b1ceffb61ff admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:00.945587 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-e9ef29d7-7f5f-4c87-8f9b-52261a0d8d9e None None] MFA Rules not processed for user `c7dc5019b26a4a8abf6436be3f1e56af`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:19:00.946923 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-e9ef29d7-7f5f-4c87-8f9b-52261a0d8d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:00.956550 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 848/1687] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:19:00 2022] PUT /identity/v3/projects/b31b2419e7d84fd68089a257a1baf479/users/2e2cb997c7f047328ec6e7837f67d29f/roles/62f19ea272b447ccaab1576cd9bf4fd9 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:19:00.959355 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f5ff81-c52a-4d07-b296-5b1ceffb61ff admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:01.048642 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:19:01.113166 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 840/1688] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:19:00 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:19:01.124067 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a388a4ed-1ed6-4716-ab82-e12d3e685468 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:01.124361 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a388a4ed-1ed6-4716-ab82-e12d3e685468 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:01.129169 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a388a4ed-1ed6-4716-ab82-e12d3e685468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:01.157958 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f5ff81-c52a-4d07-b296-5b1ceffb61ff admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:01.185629 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3680972f-f5c9-4868-a6d2-cd00ffee1325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:01.186609 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3680972f-f5c9-4868-a6d2-cd00ffee1325 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:01.186609 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3680972f-f5c9-4868-a6d2-cd00ffee1325 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:01.186849 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3680972f-f5c9-4868-a6d2-cd00ffee1325 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:01.187515 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3680972f-f5c9-4868-a6d2-cd00ffee1325 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:01.190509 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-a388a4ed-1ed6-4716-ab82-e12d3e685468 None None] MFA Rules not processed for user `c7dc5019b26a4a8abf6436be3f1e56af`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:19:01.192482 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3680972f-f5c9-4868-a6d2-cd00ffee1325 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:01.194254 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-a388a4ed-1ed6-4716-ab82-e12d3e685468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:01.228041 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 841/1689] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:19:01 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:19:01.229187 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a388a4ed-1ed6-4716-ab82-e12d3e685468 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:01.230940 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 849/1690] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:19:00 2022] GET /identity/v3/roles => generated 1420 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:19:01.232811 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-a388a4ed-1ed6-4716-ab82-e12d3e685468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:01.233164 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3680972f-f5c9-4868-a6d2-cd00ffee1325 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:01.272414 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3680972f-f5c9-4868-a6d2-cd00ffee1325 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:01.282426 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a388a4ed-1ed6-4716-ab82-e12d3e685468 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:01.285429 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e97c8c-0f5d-420a-87d9-5c627ba3f65b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:01.286145 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e97c8c-0f5d-420a-87d9-5c627ba3f65b admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:01.286451 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e97c8c-0f5d-420a-87d9-5c627ba3f65b admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:01.286727 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e97c8c-0f5d-420a-87d9-5c627ba3f65b admin admin] PATH_INFO: `/v3/projects/9a42b3ecbaf04e558979c9cf1f06e236/users/5f5c083feda3427496276f825d078d20/roles/3255f77192524a16bc405173106871fc` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:01.306341 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4bba61-2d71-428d-ad3a-ffb1d3da8c92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:01.307038 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4bba61-2d71-428d-ad3a-ffb1d3da8c92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:01.307319 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4bba61-2d71-428d-ad3a-ffb1d3da8c92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:01.307592 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4bba61-2d71-428d-ad3a-ffb1d3da8c92 admin admin] PATH_INFO: `/v3/projects/4a5aa2842b9a45fa991e41b7764abe14` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:01.312407 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4bba61-2d71-428d-ad3a-ffb1d3da8c92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a5aa2842b9a45fa991e41b7764abe14)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:01.327319 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4bba61-2d71-428d-ad3a-ffb1d3da8c92 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:01.348554 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e97c8c-0f5d-420a-87d9-5c627ba3f65b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a42b3ecbaf04e558979c9cf1f06e236, user_id=5f5c083feda3427496276f825d078d20, role_id=3255f77192524a16bc405173106871fc)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:01.355966 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e97c8c-0f5d-420a-87d9-5c627ba3f65b admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:01.493463 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 850/1691] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:19:01 2022] PUT /identity/v3/projects/9a42b3ecbaf04e558979c9cf1f06e236/users/5f5c083feda3427496276f825d078d20/roles/3255f77192524a16bc405173106871fc => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:19:01.494901 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e97c8c-0f5d-420a-87d9-5c627ba3f65b admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:01.499044 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-d0e97c8c-0f5d-420a-87d9-5c627ba3f65b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:01.531669 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-0c4bba61-2d71-428d-ad3a-ffb1d3da8c92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a5aa2842b9a45fa991e41b7764abe14', 'request_id': 'req-0c4bba61-2d71-428d-ad3a-ffb1d3da8c92'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:01.607266 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-0c4bba61-2d71-428d-ad3a-ffb1d3da8c92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a5aa2842b9a45fa991e41b7764abe14', 'request_id': 'req-0c4bba61-2d71-428d-ad3a-ffb1d3da8c92'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:01.617850 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-0c4bba61-2d71-428d-ad3a-ffb1d3da8c92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a5aa2842b9a45fa991e41b7764abe14', 'request_id': 'req-0c4bba61-2d71-428d-ad3a-ffb1d3da8c92'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:01.657889 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-0c4bba61-2d71-428d-ad3a-ffb1d3da8c92 admin admin] The token cache is being invalidate because project 4a5aa2842b9a45fa991e41b7764abe14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81995) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:19:01.658547 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-0c4bba61-2d71-428d-ad3a-ffb1d3da8c92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:01.659822 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-0c4bba61-2d71-428d-ad3a-ffb1d3da8c92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:01.666504 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 842/1692] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:19:01 2022] DELETE /identity/v3/projects/4a5aa2842b9a45fa991e41b7764abe14 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:19:01.668249 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96779da-39fc-43f4-a500-9cd48665d3eb None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:01.673138 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96779da-39fc-43f4-a500-9cd48665d3eb None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:01.673138 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96779da-39fc-43f4-a500-9cd48665d3eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:01.710050 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-f96779da-39fc-43f4-a500-9cd48665d3eb None None] MFA Rules not processed for user `5f5c083feda3427496276f825d078d20`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:19:01.710306 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-f96779da-39fc-43f4-a500-9cd48665d3eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:01.732083 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e97c8c-0f5d-420a-87d9-5c627ba3f65b admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:01.877613 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 843/1693] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:19:01 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:19:01.888158 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79dce2c-c435-4792-bc1d-a8e9d040c808 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:01.888504 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79dce2c-c435-4792-bc1d-a8e9d040c808 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:01.888788 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79dce2c-c435-4792-bc1d-a8e9d040c808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:01.913108 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f2f7ad-2f6d-472c-9fad-03222d7f18bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cd171bf4df94f8fab397e42835feeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:01.913644 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f2f7ad-2f6d-472c-9fad-03222d7f18bc service nova] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:01.913936 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f2f7ad-2f6d-472c-9fad-03222d7f18bc service nova] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:01.914222 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f2f7ad-2f6d-472c-9fad-03222d7f18bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:01.917174 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-c79dce2c-c435-4792-bc1d-a8e9d040c808 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:19:01.919025 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-c79dce2c-c435-4792-bc1d-a8e9d040c808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:01.926712 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-a2f2f7ad-2f6d-472c-9fad-03222d7f18bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:02.064536 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 844/1694] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:19:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:19:02.066610 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79dce2c-c435-4792-bc1d-a8e9d040c808 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:02.086392 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:19:02.086743 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f2f7ad-2f6d-472c-9fad-03222d7f18bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:02.087382 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-c79dce2c-c435-4792-bc1d-a8e9d040c808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:02.088369 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f2f7ad-2f6d-472c-9fad-03222d7f18bc service nova] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:02.114443 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 851/1695] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:19:01 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 620 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:19:02.131617 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0b1d4d-38ac-438b-b194-caaf9f0552f2 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:02.139506 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0b1d4d-38ac-438b-b194-caaf9f0552f2 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:02.139506 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0b1d4d-38ac-438b-b194-caaf9f0552f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:02.139858 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79dce2c-c435-4792-bc1d-a8e9d040c808 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:02.149378 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc447dc-6685-499e-9a06-30fada7e0800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:02.151870 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc447dc-6685-499e-9a06-30fada7e0800 admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:02.152334 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc447dc-6685-499e-9a06-30fada7e0800 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:02.153904 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc447dc-6685-499e-9a06-30fada7e0800 admin admin] PATH_INFO: `/v3/projects` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:02.155150 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc447dc-6685-499e-9a06-30fada7e0800 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:02.157403 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc447dc-6685-499e-9a06-30fada7e0800 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:02.169816 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:19:02.251433 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-2c0b1d4d-38ac-438b-b194-caaf9f0552f2 None None] MFA Rules not processed for user `83e90afadb164c82bc93c47099c14460`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:19:02.256592 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-2c0b1d4d-38ac-438b-b194-caaf9f0552f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:02.277057 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 845/1696] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:19:02 2022] POST /identity/v3/projects => generated 409 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:19:02.277222 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0903cd7d-73b9-4092-ab3e-ab854d372c4a None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:02.277849 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0903cd7d-73b9-4092-ab3e-ab854d372c4a None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:02.280680 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0903cd7d-73b9-4092-ab3e-ab854d372c4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:02.323305 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-0903cd7d-73b9-4092-ab3e-ab854d372c4a None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:19:02.331714 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-0903cd7d-73b9-4092-ab3e-ab854d372c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:02.451106 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 852/1697] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:19:02 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:19:02.454365 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c908c31-27ee-4ca1-80e5-7f314a654607 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:02.454655 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c908c31-27ee-4ca1-80e5-7f314a654607 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:02.454933 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c908c31-27ee-4ca1-80e5-7f314a654607 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:02.462058 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 853/1698] 10.222.0.58 () {54 vars in 961 bytes} [Thu Aug 25 03:19:02 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:19:02.464505 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c908c31-27ee-4ca1-80e5-7f314a654607 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:02.471573 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 846/1699] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:19:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:19:02.472888 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b335a0-a5ab-4ec6-9a2e-b37ea007d9e4 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:02.473250 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b335a0-a5ab-4ec6-9a2e-b37ea007d9e4 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:02.473630 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b335a0-a5ab-4ec6-9a2e-b37ea007d9e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:02.500476 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c908c31-27ee-4ca1-80e5-7f314a654607 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:02.500791 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-b8b335a0-a5ab-4ec6-9a2e-b37ea007d9e4 None None] MFA Rules not processed for user `2b692edd3e8c41289cc42e1afb4a6dd8`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:19:02.503182 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-b8b335a0-a5ab-4ec6-9a2e-b37ea007d9e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:02.511040 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f934d9-b7fe-4c78-a12c-53f5aff38019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:02.515404 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f934d9-b7fe-4c78-a12c-53f5aff38019 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:02.515729 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f934d9-b7fe-4c78-a12c-53f5aff38019 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:02.516012 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f934d9-b7fe-4c78-a12c-53f5aff38019 admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:02.517250 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f934d9-b7fe-4c78-a12c-53f5aff38019 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:02.522044 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f934d9-b7fe-4c78-a12c-53f5aff38019 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:02.633030 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 847/1700] 10.222.0.58 () {58 vars in 1058 bytes} [Thu Aug 25 03:19:02 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:19:02.634369 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b335a0-a5ab-4ec6-9a2e-b37ea007d9e4 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:02.647316 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b335a0-a5ab-4ec6-9a2e-b37ea007d9e4 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:02.653373 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2718ae5-0d09-4e7b-a855-ec83fea636eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:02.654176 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2718ae5-0d09-4e7b-a855-ec83fea636eb admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:02.654725 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2718ae5-0d09-4e7b-a855-ec83fea636eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:02.655098 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2718ae5-0d09-4e7b-a855-ec83fea636eb admin admin] PATH_INFO: `/v3/domains` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:02.656225 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2718ae5-0d09-4e7b-a855-ec83fea636eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:19:02.656670 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2718ae5-0d09-4e7b-a855-ec83fea636eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:02.658501 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2718ae5-0d09-4e7b-a855-ec83fea636eb admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:02.675395 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 848/1701] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Aug 25 03:19:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:19:02.675395 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2718ae5-0d09-4e7b-a855-ec83fea636eb admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:02.694314 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 854/1702] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:19:02 2022] POST /identity/v3/users => generated 311 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:19:02.697333 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2718ae5-0d09-4e7b-a855-ec83fea636eb admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:02.698203 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f934d9-b7fe-4c78-a12c-53f5aff38019 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:02.704194 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeebd98-11f0-49c1-931a-c85069746723 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b692edd3e8c41289cc42e1afb4a6dd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:02.704953 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeebd98-11f0-49c1-931a-c85069746723 service placement] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:02.705328 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeebd98-11f0-49c1-931a-c85069746723 service placement] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:02.706086 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeebd98-11f0-49c1-931a-c85069746723 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:02.709433 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-beeebd98-11f0-49c1-931a-c85069746723 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:02.723845 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f934d9-b7fe-4c78-a12c-53f5aff38019 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:02.727147 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beeebd98-11f0-49c1-931a-c85069746723 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:02.728974 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beeebd98-11f0-49c1-931a-c85069746723 service placement] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:02.735038 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892d37c7-3ba0-4aff-96c4-a3b57c32a3c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:02.735782 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892d37c7-3ba0-4aff-96c4-a3b57c32a3c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:02.736066 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892d37c7-3ba0-4aff-96c4-a3b57c32a3c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:02.736341 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892d37c7-3ba0-4aff-96c4-a3b57c32a3c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:02.737149 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892d37c7-3ba0-4aff-96c4-a3b57c32a3c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:02.739707 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892d37c7-3ba0-4aff-96c4-a3b57c32a3c5 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:02.753521 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 849/1703] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:19:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:19:02.758187 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeebd98-11f0-49c1-931a-c85069746723 service placement] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:02.773666 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeebd98-11f0-49c1-931a-c85069746723 service placement] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:02.778976 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d519108-69f0-46be-9da7-51f91d090f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:02.779743 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d519108-69f0-46be-9da7-51f91d090f45 admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:02.780034 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d519108-69f0-46be-9da7-51f91d090f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:02.780311 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d519108-69f0-46be-9da7-51f91d090f45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:02.780960 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d519108-69f0-46be-9da7-51f91d090f45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:02.782728 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d519108-69f0-46be-9da7-51f91d090f45 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:02.801011 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 850/1704] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:19:02 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:19:02.801946 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d519108-69f0-46be-9da7-51f91d090f45 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:02.813212 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d519108-69f0-46be-9da7-51f91d090f45 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:02.818906 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57bb38f-571f-4446-be02-f986e3713cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:02.826931 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57bb38f-571f-4446-be02-f986e3713cb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:02.826931 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57bb38f-571f-4446-be02-f986e3713cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:02.826931 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57bb38f-571f-4446-be02-f986e3713cb2 admin admin] PATH_INFO: `/v3/projects/2c8c1fad8d25471986efab70fcaed338/users/d248205be76440399cfdee05d8e508c2/roles/62f19ea272b447ccaab1576cd9bf4fd9` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:02.837714 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57bb38f-571f-4446-be02-f986e3713cb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c8c1fad8d25471986efab70fcaed338, user_id=d248205be76440399cfdee05d8e508c2, role_id=62f19ea272b447ccaab1576cd9bf4fd9)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:02.840523 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57bb38f-571f-4446-be02-f986e3713cb2 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:02.868129 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 855/1705] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:19:02 2022] POST /identity/v3/projects => generated 375 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:19:02.874582 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892d37c7-3ba0-4aff-96c4-a3b57c32a3c5 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:02.955540 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 851/1706] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:19:02 2022] PUT /identity/v3/projects/2c8c1fad8d25471986efab70fcaed338/users/d248205be76440399cfdee05d8e508c2/roles/62f19ea272b447ccaab1576cd9bf4fd9 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:19:02.955540 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c05e314-1d76-4979-ace4-91e013e9a70a None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:02.955540 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c05e314-1d76-4979-ace4-91e013e9a70a None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:02.955812 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c05e314-1d76-4979-ace4-91e013e9a70a None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:02.961733 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 852/1707] 10.222.0.58 () {54 vars in 912 bytes} [Thu Aug 25 03:19:02 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:19:02.964490 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c05e314-1d76-4979-ace4-91e013e9a70a None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:02.967218 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-1c05e314-1d76-4979-ace4-91e013e9a70a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:03.046329 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892d37c7-3ba0-4aff-96c4-a3b57c32a3c5 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:03.221847 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb297eb-d48f-4382-af15-49dd01fd8aa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:03.222588 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb297eb-d48f-4382-af15-49dd01fd8aa6 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:03.222799 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb297eb-d48f-4382-af15-49dd01fd8aa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:03.223078 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb297eb-d48f-4382-af15-49dd01fd8aa6 admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:03.223713 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb297eb-d48f-4382-af15-49dd01fd8aa6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:03.228483 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb297eb-d48f-4382-af15-49dd01fd8aa6 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:03.300298 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c05e314-1d76-4979-ace4-91e013e9a70a None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:03.312604 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc5f5c5-6ad7-493b-bd06-7068c76df381 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cd171bf4df94f8fab397e42835feeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:03.313523 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc5f5c5-6ad7-493b-bd06-7068c76df381 service nova] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:03.313967 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc5f5c5-6ad7-493b-bd06-7068c76df381 service nova] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:03.314425 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc5f5c5-6ad7-493b-bd06-7068c76df381 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:03.349264 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:19:03.384040 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 856/1708] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:19:02 2022] POST /identity/v3/users => generated 288 bytes in 510 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:19:03.385665 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb297eb-d48f-4382-af15-49dd01fd8aa6 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:03.419593 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb297eb-d48f-4382-af15-49dd01fd8aa6 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:03.429088 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982ca9b5-7a20-44d8-a03f-862db0361c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:03.431634 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982ca9b5-7a20-44d8-a03f-862db0361c1d admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:03.431634 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982ca9b5-7a20-44d8-a03f-862db0361c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:03.432179 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982ca9b5-7a20-44d8-a03f-862db0361c1d admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:03.433008 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982ca9b5-7a20-44d8-a03f-862db0361c1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:03.434854 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982ca9b5-7a20-44d8-a03f-862db0361c1d admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:03.445648 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc5f5c5-6ad7-493b-bd06-7068c76df381 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:03.447718 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc5f5c5-6ad7-493b-bd06-7068c76df381 service nova] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:03.453345 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 857/1709] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:19:03 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:19:03.454912 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c9765d-2cbe-4e71-a1c7-d1872b5f00de None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:03.455351 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c9765d-2cbe-4e71-a1c7-d1872b5f00de None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:03.455774 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c9765d-2cbe-4e71-a1c7-d1872b5f00de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:03.469668 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 853/1710] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:19:02 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 506 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:19:03.469668 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc5f5c5-6ad7-493b-bd06-7068c76df381 service nova] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:03.484087 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc5f5c5-6ad7-493b-bd06-7068c76df381 service nova] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:03.490575 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3908b8-3c41-41dc-86db-6c206d46207e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:03.491638 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3908b8-3c41-41dc-86db-6c206d46207e admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:03.491926 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3908b8-3c41-41dc-86db-6c206d46207e admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:03.492201 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3908b8-3c41-41dc-86db-6c206d46207e admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:03.492969 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3908b8-3c41-41dc-86db-6c206d46207e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:03.494871 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3908b8-3c41-41dc-86db-6c206d46207e admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:03.501937 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-52c9765d-2cbe-4e71-a1c7-d1872b5f00de None None] MFA Rules not processed for user `a0a1a7f091774842907fd08de15e8058`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:19:03.504956 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-52c9765d-2cbe-4e71-a1c7-d1872b5f00de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:03.514856 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 854/1711] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:19:03 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:19:03.514856 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3908b8-3c41-41dc-86db-6c206d46207e admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:03.569968 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3908b8-3c41-41dc-86db-6c206d46207e admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:03.589901 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57f3744-c3ec-4086-8204-dbb1e8f74743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:03.591172 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57f3744-c3ec-4086-8204-dbb1e8f74743 admin admin] REQUEST_METHOD: `PUT` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:03.591453 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57f3744-c3ec-4086-8204-dbb1e8f74743 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:03.591726 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57f3744-c3ec-4086-8204-dbb1e8f74743 admin admin] PATH_INFO: `/v3/projects/2c8c1fad8d25471986efab70fcaed338/users/d248205be76440399cfdee05d8e508c2/roles/62f19ea272b447ccaab1576cd9bf4fd9` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:03.596429 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57f3744-c3ec-4086-8204-dbb1e8f74743 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c8c1fad8d25471986efab70fcaed338, user_id=d248205be76440399cfdee05d8e508c2, role_id=62f19ea272b447ccaab1576cd9bf4fd9)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:03.601133 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57f3744-c3ec-4086-8204-dbb1e8f74743 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:03.646417 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 855/1712] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:19:03 2022] PUT /identity/v3/projects/2c8c1fad8d25471986efab70fcaed338/users/d248205be76440399cfdee05d8e508c2/roles/62f19ea272b447ccaab1576cd9bf4fd9 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:19:03.648292 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57f3744-c3ec-4086-8204-dbb1e8f74743 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:03.689459 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 858/1713] 10.222.0.58 () {58 vars in 1009 bytes} [Thu Aug 25 03:19:03 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:19:03.691423 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f31f2d2-3739-4c42-9159-daaf40eff3d5 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:03.691950 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f31f2d2-3739-4c42-9159-daaf40eff3d5 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:03.692370 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f31f2d2-3739-4c42-9159-daaf40eff3d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:03.699093 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:19:03.749328 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-5f31f2d2-3739-4c42-9159-daaf40eff3d5 None None] MFA Rules not processed for user `d248205be76440399cfdee05d8e508c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:19:03.754940 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-5f31f2d2-3739-4c42-9159-daaf40eff3d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:03.838604 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57f3744-c3ec-4086-8204-dbb1e8f74743 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:03.845093 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8f84da-f3f8-4fda-bf03-8110b168a747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:03.846080 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8f84da-f3f8-4fda-bf03-8110b168a747 admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:03.846757 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8f84da-f3f8-4fda-bf03-8110b168a747 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:03.847135 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8f84da-f3f8-4fda-bf03-8110b168a747 admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:03.847951 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8f84da-f3f8-4fda-bf03-8110b168a747 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:03.852151 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8f84da-f3f8-4fda-bf03-8110b168a747 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:03.873503 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 856/1714] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:19:03 2022] GET /identity/v3/roles => generated 1420 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:19:03.874601 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8f84da-f3f8-4fda-bf03-8110b168a747 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:03.883784 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-8d8f84da-f3f8-4fda-bf03-8110b168a747 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:03.954165 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 859/1715] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:19:03 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:19:03.961281 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b6171e-1215-4642-ad7e-738b41b488a4 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:03.961281 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b6171e-1215-4642-ad7e-738b41b488a4 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:03.961281 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b6171e-1215-4642-ad7e-738b41b488a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:04.026369 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-62b6171e-1215-4642-ad7e-738b41b488a4 None None] MFA Rules not processed for user `2e2cb997c7f047328ec6e7837f67d29f`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:19:04.030196 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-62b6171e-1215-4642-ad7e-738b41b488a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:04.041572 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8f84da-f3f8-4fda-bf03-8110b168a747 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:04.056581 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2fe1df-11fb-41e2-87b7-09d15fc80e79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:04.061214 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2fe1df-11fb-41e2-87b7-09d15fc80e79 service neutron] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:04.061214 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2fe1df-11fb-41e2-87b7-09d15fc80e79 service neutron] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:04.061214 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2fe1df-11fb-41e2-87b7-09d15fc80e79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:04.090203 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2fe1df-11fb-41e2-87b7-09d15fc80e79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:04.094190 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2fe1df-11fb-41e2-87b7-09d15fc80e79 service neutron] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:04.112933 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 857/1716] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:19:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:19:04.114869 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2fe1df-11fb-41e2-87b7-09d15fc80e79 service neutron] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:04.146237 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2fe1df-11fb-41e2-87b7-09d15fc80e79 service neutron] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:04.158416 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d5da96-c0c4-461d-8a14-34d30fbedc0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:04.159098 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d5da96-c0c4-461d-8a14-34d30fbedc0b admin admin] REQUEST_METHOD: `PUT` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:04.159379 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d5da96-c0c4-461d-8a14-34d30fbedc0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:04.159661 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d5da96-c0c4-461d-8a14-34d30fbedc0b admin admin] PATH_INFO: `/v3/projects/da4d7a1bdb68408182e7982ce684907c/users/5fb7ef699a044b8d956a0c02a53777a0/roles/3255f77192524a16bc405173106871fc` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:04.190607 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d5da96-c0c4-461d-8a14-34d30fbedc0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=da4d7a1bdb68408182e7982ce684907c, user_id=5fb7ef699a044b8d956a0c02a53777a0, role_id=3255f77192524a16bc405173106871fc)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:04.193375 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d5da96-c0c4-461d-8a14-34d30fbedc0b admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:04.206692 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 860/1717] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:19:03 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:19:04.208828 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b6171e-1215-4642-ad7e-738b41b488a4 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:04.212112 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-62b6171e-1215-4642-ad7e-738b41b488a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:04.276098 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 858/1718] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:19:04 2022] PUT /identity/v3/projects/da4d7a1bdb68408182e7982ce684907c/users/5fb7ef699a044b8d956a0c02a53777a0/roles/3255f77192524a16bc405173106871fc => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:19:04.286353 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dd0556-2402-4e64-b41a-be400780b962 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:04.287112 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dd0556-2402-4e64-b41a-be400780b962 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:04.287537 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dd0556-2402-4e64-b41a-be400780b962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:04.348094 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-b6dd0556-2402-4e64-b41a-be400780b962 None None] MFA Rules not processed for user `5fb7ef699a044b8d956a0c02a53777a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:19:04.349500 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-b6dd0556-2402-4e64-b41a-be400780b962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:04.389413 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b6171e-1215-4642-ad7e-738b41b488a4 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:04.503850 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 859/1719] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:19:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:19:04.507864 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dd0556-2402-4e64-b41a-be400780b962 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:04.563053 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6150bd-15d0-4f20-83e5-d9c2dbaf7eec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60193ba9e1c14df3bc9eebcc6f28f2d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:04.564025 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6150bd-15d0-4f20-83e5-d9c2dbaf7eec service cinder] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:04.564437 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6150bd-15d0-4f20-83e5-d9c2dbaf7eec service cinder] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:04.564878 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6150bd-15d0-4f20-83e5-d9c2dbaf7eec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:04.697975 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dd0556-2402-4e64-b41a-be400780b962 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:04.714006 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e33b737-4faa-462c-93c5-f00104cb5bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:04.714771 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e33b737-4faa-462c-93c5-f00104cb5bac admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:04.715460 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:19:04.717383 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e33b737-4faa-462c-93c5-f00104cb5bac admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:04.717690 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e33b737-4faa-462c-93c5-f00104cb5bac admin admin] PATH_INFO: `/v3/projects` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:04.718382 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e33b737-4faa-462c-93c5-f00104cb5bac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:04.724931 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e33b737-4faa-462c-93c5-f00104cb5bac admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:04.796606 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6150bd-15d0-4f20-83e5-d9c2dbaf7eec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:04.798076 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6150bd-15d0-4f20-83e5-d9c2dbaf7eec service cinder] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:04.820543 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 861/1720] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:19:04 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 613 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:19:04.902025 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 860/1721] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:19:04 2022] POST /identity/v3/projects => generated 375 bytes in 394 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:19:04.905931 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6150bd-15d0-4f20-83e5-d9c2dbaf7eec service cinder] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:04.925214 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6150bd-15d0-4f20-83e5-d9c2dbaf7eec service cinder] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:04.979191 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:19:05.086928 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66c7275-83bd-40df-b012-bcffc14742ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:05.087555 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66c7275-83bd-40df-b012-bcffc14742ac admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:05.087834 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66c7275-83bd-40df-b012-bcffc14742ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:05.088230 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66c7275-83bd-40df-b012-bcffc14742ac admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:05.088829 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66c7275-83bd-40df-b012-bcffc14742ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:05.094929 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66c7275-83bd-40df-b012-bcffc14742ac admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:05.196485 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 862/1722] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:19:04 2022] POST /identity/v3/users => generated 294 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:19:05.201054 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e33b737-4faa-462c-93c5-f00104cb5bac admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:05.218370 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e33b737-4faa-462c-93c5-f00104cb5bac admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:05.229445 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87661294-f9fd-4f0a-95b4-c2c79cad45be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:05.231194 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87661294-f9fd-4f0a-95b4-c2c79cad45be admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:05.231490 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87661294-f9fd-4f0a-95b4-c2c79cad45be admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:05.231772 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87661294-f9fd-4f0a-95b4-c2c79cad45be admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:05.234932 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87661294-f9fd-4f0a-95b4-c2c79cad45be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:05.242682 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87661294-f9fd-4f0a-95b4-c2c79cad45be admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:05.282821 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 861/1723] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:19:05 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:19:05.283804 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66c7275-83bd-40df-b012-bcffc14742ac admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:05.297627 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66c7275-83bd-40df-b012-bcffc14742ac admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:05.331877 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263df16c-0860-4321-8cfe-ccc256467579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:05.331877 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263df16c-0860-4321-8cfe-ccc256467579 admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:05.331877 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263df16c-0860-4321-8cfe-ccc256467579 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:05.331877 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263df16c-0860-4321-8cfe-ccc256467579 admin admin] PATH_INFO: `/v3/projects/ebda46e793bf49599fc83fe4fb7a337d/users/6d673ef3236643378c2e5101c354d59b/roles/62f19ea272b447ccaab1576cd9bf4fd9` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:05.374063 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263df16c-0860-4321-8cfe-ccc256467579 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebda46e793bf49599fc83fe4fb7a337d, user_id=6d673ef3236643378c2e5101c354d59b, role_id=62f19ea272b447ccaab1576cd9bf4fd9)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:05.378652 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263df16c-0860-4321-8cfe-ccc256467579 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:05.477983 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 863/1724] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:19:05 2022] PUT /identity/v3/projects/ebda46e793bf49599fc83fe4fb7a337d/users/6d673ef3236643378c2e5101c354d59b/roles/62f19ea272b447ccaab1576cd9bf4fd9 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:19:05.482638 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87661294-f9fd-4f0a-95b4-c2c79cad45be admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:05.494794 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87661294-f9fd-4f0a-95b4-c2c79cad45be admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:05.539097 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263df16c-0860-4321-8cfe-ccc256467579 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:05.541363 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-263df16c-0860-4321-8cfe-ccc256467579 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:05.648370 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b513759e-b67a-4bd8-afa3-60ebae974675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:05.649791 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b513759e-b67a-4bd8-afa3-60ebae974675 admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:05.650247 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b513759e-b67a-4bd8-afa3-60ebae974675 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:05.651840 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b513759e-b67a-4bd8-afa3-60ebae974675 admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:05.652852 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b513759e-b67a-4bd8-afa3-60ebae974675 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:05.656870 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b513759e-b67a-4bd8-afa3-60ebae974675 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:05.677460 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263df16c-0860-4321-8cfe-ccc256467579 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:05.691811 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59aae53-5de4-46ec-9f02-bce265bbcc21 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60193ba9e1c14df3bc9eebcc6f28f2d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:05.693070 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 862/1725] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:19:05 2022] GET /identity/v3/roles => generated 1420 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:19:05.693569 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59aae53-5de4-46ec-9f02-bce265bbcc21 service cinder] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:05.694045 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59aae53-5de4-46ec-9f02-bce265bbcc21 service cinder] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:05.694529 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59aae53-5de4-46ec-9f02-bce265bbcc21 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:05.696827 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b513759e-b67a-4bd8-afa3-60ebae974675 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:05.719218 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b513759e-b67a-4bd8-afa3-60ebae974675 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:05.729360 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd02072-8f7e-43a7-91d7-840234638bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:05.730051 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd02072-8f7e-43a7-91d7-840234638bc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:05.732615 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd02072-8f7e-43a7-91d7-840234638bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:05.732615 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd02072-8f7e-43a7-91d7-840234638bc8 admin admin] PATH_INFO: `/v3/projects/ebda46e793bf49599fc83fe4fb7a337d/users/6d673ef3236643378c2e5101c354d59b/roles/62f19ea272b447ccaab1576cd9bf4fd9` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:05.737418 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd02072-8f7e-43a7-91d7-840234638bc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebda46e793bf49599fc83fe4fb7a337d, user_id=6d673ef3236643378c2e5101c354d59b, role_id=62f19ea272b447ccaab1576cd9bf4fd9)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:05.742002 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd02072-8f7e-43a7-91d7-840234638bc8 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:05.756524 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:19:05.766616 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 863/1726] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:19:05 2022] PUT /identity/v3/projects/ebda46e793bf49599fc83fe4fb7a337d/users/6d673ef3236643378c2e5101c354d59b/roles/62f19ea272b447ccaab1576cd9bf4fd9 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:19:05.776444 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc84f3ff-d32f-4b44-ae92-c75ca74efc77 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:05.776980 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc84f3ff-d32f-4b44-ae92-c75ca74efc77 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:05.777542 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc84f3ff-d32f-4b44-ae92-c75ca74efc77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:05.854256 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59aae53-5de4-46ec-9f02-bce265bbcc21 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:05.856056 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59aae53-5de4-46ec-9f02-bce265bbcc21 service cinder] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:05.875956 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 864/1727] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:19:05 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 25 03:19:05.877645 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfff1251-efc1-4ba1-b88b-dcb2f0229972 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:05.878200 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfff1251-efc1-4ba1-b88b-dcb2f0229972 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:05.878786 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfff1251-efc1-4ba1-b88b-dcb2f0229972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:05.900469 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-cc84f3ff-d32f-4b44-ae92-c75ca74efc77 None None] MFA Rules not processed for user `6d673ef3236643378c2e5101c354d59b`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:19:05.905136 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-cc84f3ff-d32f-4b44-ae92-c75ca74efc77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:05.916862 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-dfff1251-efc1-4ba1-b88b-dcb2f0229972 None None] MFA Rules not processed for user `c7dc5019b26a4a8abf6436be3f1e56af`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:19:05.918463 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-dfff1251-efc1-4ba1-b88b-dcb2f0229972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:05.984351 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:19:06.042644 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:19:06.123963 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 865/1728] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:19:05 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:19:06.163488 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 864/1729] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:19:05 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 385 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:19:06.286144 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f169fcd4-65d9-40ba-ba1d-d47c2230a568 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:06.286745 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f169fcd4-65d9-40ba-ba1d-d47c2230a568 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:06.287248 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f169fcd4-65d9-40ba-ba1d-d47c2230a568 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:06.290042 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 866/1730] 10.222.0.58 () {54 vars in 921 bytes} [Thu Aug 25 03:19:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:19:06.320779 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96ae4ea-8d1a-422a-8f1a-d5bbfd8aa706 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:06.321066 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96ae4ea-8d1a-422a-8f1a-d5bbfd8aa706 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:06.321586 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96ae4ea-8d1a-422a-8f1a-d5bbfd8aa706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:06.354707 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-b96ae4ea-8d1a-422a-8f1a-d5bbfd8aa706 None None] MFA Rules not processed for user `2b692edd3e8c41289cc42e1afb4a6dd8`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:19:06.357739 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-b96ae4ea-8d1a-422a-8f1a-d5bbfd8aa706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:06.408753 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f169fcd4-65d9-40ba-ba1d-d47c2230a568 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:06.543152 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 865/1731] 10.222.0.58 () {58 vars in 1018 bytes} [Thu Aug 25 03:19:06 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:19:06.573756 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96ae4ea-8d1a-422a-8f1a-d5bbfd8aa706 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:06.573756 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-b96ae4ea-8d1a-422a-8f1a-d5bbfd8aa706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:06.574708 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f169fcd4-65d9-40ba-ba1d-d47c2230a568 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:06.577131 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0adc476-cfc2-49de-94fa-05f3d439d6ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60193ba9e1c14df3bc9eebcc6f28f2d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:06.577933 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0adc476-cfc2-49de-94fa-05f3d439d6ac service cinder] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:06.578646 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0adc476-cfc2-49de-94fa-05f3d439d6ac service cinder] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:06.579202 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0adc476-cfc2-49de-94fa-05f3d439d6ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:06.595308 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:19:06.600801 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96ae4ea-8d1a-422a-8f1a-d5bbfd8aa706 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:06.616411 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6206e5e3-d533-44a7-a164-e175aee60c26 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b692edd3e8c41289cc42e1afb4a6dd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:06.617319 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6206e5e3-d533-44a7-a164-e175aee60c26 service placement] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:06.617750 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6206e5e3-d533-44a7-a164-e175aee60c26 service placement] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:06.618164 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6206e5e3-d533-44a7-a164-e175aee60c26 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:06.651855 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6206e5e3-d533-44a7-a164-e175aee60c26 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:06.654589 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6206e5e3-d533-44a7-a164-e175aee60c26 service placement] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:06.668353 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0adc476-cfc2-49de-94fa-05f3d439d6ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:06.670764 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0adc476-cfc2-49de-94fa-05f3d439d6ac service cinder] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:06.683095 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 867/1732] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:19:06 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:19:06.683959 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 866/1733] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:19:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 25 03:19:06.726496 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:19:06.974580 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0adc476-cfc2-49de-94fa-05f3d439d6ac service cinder] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:06.976166 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-f0adc476-cfc2-49de-94fa-05f3d439d6ac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:07.223866 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0adc476-cfc2-49de-94fa-05f3d439d6ac service cinder] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:07.229640 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85959c0f-c82a-46b1-bb61-0d7c09751e46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:07.230340 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85959c0f-c82a-46b1-bb61-0d7c09751e46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:07.230625 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85959c0f-c82a-46b1-bb61-0d7c09751e46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:07.230914 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85959c0f-c82a-46b1-bb61-0d7c09751e46 admin admin] PATH_INFO: `/v3/users/680dbbbc2e9249cb865ddc540461ac28` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:07.240957 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85959c0f-c82a-46b1-bb61-0d7c09751e46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=680dbbbc2e9249cb865ddc540461ac28)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:07.243524 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85959c0f-c82a-46b1-bb61-0d7c09751e46 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:07.570882 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-85959c0f-c82a-46b1-bb61-0d7c09751e46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '680dbbbc2e9249cb865ddc540461ac28'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:07.675262 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-85959c0f-c82a-46b1-bb61-0d7c09751e46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '680dbbbc2e9249cb865ddc540461ac28'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:07.676342 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-85959c0f-c82a-46b1-bb61-0d7c09751e46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '680dbbbc2e9249cb865ddc540461ac28'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:07.688701 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-85959c0f-c82a-46b1-bb61-0d7c09751e46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '680dbbbc2e9249cb865ddc540461ac28'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:07.737454 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-85959c0f-c82a-46b1-bb61-0d7c09751e46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '680dbbbc2e9249cb865ddc540461ac28'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:07.754073 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 868/1734] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:19:06 2022] DELETE /identity/v3/users/680dbbbc2e9249cb865ddc540461ac28 => generated 0 bytes in 777 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:19:07.756121 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1805f655-fb92-4fe8-8bee-6951494de33d None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:07.757011 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1805f655-fb92-4fe8-8bee-6951494de33d None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:07.757350 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1805f655-fb92-4fe8-8bee-6951494de33d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:07.810385 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-1805f655-fb92-4fe8-8bee-6951494de33d None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:19:07.811813 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-1805f655-fb92-4fe8-8bee-6951494de33d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:07.973838 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 867/1735] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:19:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:19:07.984906 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85959c0f-c82a-46b1-bb61-0d7c09751e46 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:07.986423 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-85959c0f-c82a-46b1-bb61-0d7c09751e46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:08.037297 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:19:08.087548 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85959c0f-c82a-46b1-bb61-0d7c09751e46 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:08.093882 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e201c1-abba-4810-af36-382f7bdcba58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:08.095539 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e201c1-abba-4810-af36-382f7bdcba58 service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:08.096067 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e201c1-abba-4810-af36-382f7bdcba58 service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:08.096639 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e201c1-abba-4810-af36-382f7bdcba58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:08.132643 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e201c1-abba-4810-af36-382f7bdcba58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:08.137698 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e201c1-abba-4810-af36-382f7bdcba58 service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:08.168313 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 869/1736] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:19:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:19:08.191175 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2afd03-d124-416b-8a95-1cbd50a584b7 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:08.191649 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2afd03-d124-416b-8a95-1cbd50a584b7 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:08.192087 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2afd03-d124-416b-8a95-1cbd50a584b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:08.278506 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-dd2afd03-d124-416b-8a95-1cbd50a584b7 None None] MFA Rules not processed for user `712c35b765b64919aa3eaaf02892590d`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:19:08.279802 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-dd2afd03-d124-416b-8a95-1cbd50a584b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:08.435761 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:19:08.448062 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 868/1737] 10.222.0.58 () {58 vars in 1009 bytes} [Thu Aug 25 03:19:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:19:08.570022 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e201c1-abba-4810-af36-382f7bdcba58 service neutron] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:08.572685 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-04e201c1-abba-4810-af36-382f7bdcba58 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:08.591259 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e201c1-abba-4810-af36-382f7bdcba58 service neutron] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:08.630883 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a404c357-9f43-4d0d-88f9-01b13ecd8e46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:08.632147 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a404c357-9f43-4d0d-88f9-01b13ecd8e46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:08.632670 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a404c357-9f43-4d0d-88f9-01b13ecd8e46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:08.634007 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a404c357-9f43-4d0d-88f9-01b13ecd8e46 admin admin] PATH_INFO: `/v3/users/49405a748ab54056ba8ba9f996381135` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:08.646491 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a404c357-9f43-4d0d-88f9-01b13ecd8e46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49405a748ab54056ba8ba9f996381135)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:08.649191 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a404c357-9f43-4d0d-88f9-01b13ecd8e46 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:08.883864 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2afd03-d124-416b-8a95-1cbd50a584b7 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:08.885402 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-dd2afd03-d124-416b-8a95-1cbd50a584b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:08.979909 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-a404c357-9f43-4d0d-88f9-01b13ecd8e46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49405a748ab54056ba8ba9f996381135'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:09.111809 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-a404c357-9f43-4d0d-88f9-01b13ecd8e46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49405a748ab54056ba8ba9f996381135'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:09.112497 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-a404c357-9f43-4d0d-88f9-01b13ecd8e46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49405a748ab54056ba8ba9f996381135'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:09.121186 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-a404c357-9f43-4d0d-88f9-01b13ecd8e46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49405a748ab54056ba8ba9f996381135'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:09.166144 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2afd03-d124-416b-8a95-1cbd50a584b7 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:09.178147 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f093e6eb-e8ec-4604-80c8-be59b9a91870 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5047e1a7cd2d416d90b2d8c5a6596522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:09.178648 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f093e6eb-e8ec-4604-80c8-be59b9a91870 service swift] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:09.179092 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f093e6eb-e8ec-4604-80c8-be59b9a91870 service swift] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:09.179513 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f093e6eb-e8ec-4604-80c8-be59b9a91870 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:09.184753 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-a404c357-9f43-4d0d-88f9-01b13ecd8e46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49405a748ab54056ba8ba9f996381135'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:09.196749 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f093e6eb-e8ec-4604-80c8-be59b9a91870 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:09.203672 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f093e6eb-e8ec-4604-80c8-be59b9a91870 service swift] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:09.208993 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 870/1738] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:19:08 2022] DELETE /identity/v3/users/49405a748ab54056ba8ba9f996381135 => generated 0 bytes in 639 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:19:09.226205 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 869/1739] 10.222.0.58 () {60 vars in 1384 bytes} [Thu Aug 25 03:19:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 25 03:19:09.358484 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f98900e-2ebc-4cbb-a0ec-b5cfdd5f2f96 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:09.358784 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f98900e-2ebc-4cbb-a0ec-b5cfdd5f2f96 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:09.359093 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f98900e-2ebc-4cbb-a0ec-b5cfdd5f2f96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:09.430640 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-4f98900e-2ebc-4cbb-a0ec-b5cfdd5f2f96 None None] MFA Rules not processed for user `83e90afadb164c82bc93c47099c14460`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:19:09.435116 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-4f98900e-2ebc-4cbb-a0ec-b5cfdd5f2f96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:09.681216 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 871/1740] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:19:09 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:19:09.727465 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f093e6eb-e8ec-4604-80c8-be59b9a91870 service swift] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:09.729164 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-f093e6eb-e8ec-4604-80c8-be59b9a91870 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:09.752795 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f093e6eb-e8ec-4604-80c8-be59b9a91870 service swift] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:09.869658 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfecf57-cc9f-4178-b722-91fd1e0bd118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:09.871351 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfecf57-cc9f-4178-b722-91fd1e0bd118 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:09.871351 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfecf57-cc9f-4178-b722-91fd1e0bd118 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:09.876790 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfecf57-cc9f-4178-b722-91fd1e0bd118 admin admin] PATH_INFO: `/v3/projects/a4754b68abd046f68f0f422c90a583f5` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:09.876790 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfecf57-cc9f-4178-b722-91fd1e0bd118 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4754b68abd046f68f0f422c90a583f5)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:09.878557 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfecf57-cc9f-4178-b722-91fd1e0bd118 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:10.089724 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-6dfecf57-cc9f-4178-b722-91fd1e0bd118 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4754b68abd046f68f0f422c90a583f5', 'request_id': 'req-6dfecf57-cc9f-4178-b722-91fd1e0bd118'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:10.212816 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-6dfecf57-cc9f-4178-b722-91fd1e0bd118 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4754b68abd046f68f0f422c90a583f5', 'request_id': 'req-6dfecf57-cc9f-4178-b722-91fd1e0bd118'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:10.224434 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-6dfecf57-cc9f-4178-b722-91fd1e0bd118 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4754b68abd046f68f0f422c90a583f5', 'request_id': 'req-6dfecf57-cc9f-4178-b722-91fd1e0bd118'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:10.234171 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:19:10.299812 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-6dfecf57-cc9f-4178-b722-91fd1e0bd118 admin admin] The token cache is being invalidate because project a4754b68abd046f68f0f422c90a583f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81995) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:19:10.303825 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-6dfecf57-cc9f-4178-b722-91fd1e0bd118 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:10.317115 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-6dfecf57-cc9f-4178-b722-91fd1e0bd118 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:10.323422 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 870/1741] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:19:09 2022] DELETE /identity/v3/projects/a4754b68abd046f68f0f422c90a583f5 => generated 0 bytes in 596 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:19:10.336433 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f092dc-d411-411a-b999-aa61c5fbeece None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:10.337078 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f092dc-d411-411a-b999-aa61c5fbeece None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:10.337078 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f092dc-d411-411a-b999-aa61c5fbeece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:10.391683 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-24f092dc-d411-411a-b999-aa61c5fbeece None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:19:10.395937 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-24f092dc-d411-411a-b999-aa61c5fbeece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:10.505654 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:19:10.562032 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfecf57-cc9f-4178-b722-91fd1e0bd118 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:10.563758 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-6dfecf57-cc9f-4178-b722-91fd1e0bd118 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:10.639841 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 872/1742] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:19:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:19:10.642041 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfecf57-cc9f-4178-b722-91fd1e0bd118 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:10.646604 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e876fd-cc02-44b5-abfd-43d6e8bed61b None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:10.647366 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e876fd-cc02-44b5-abfd-43d6e8bed61b None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:10.655942 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22b58c1-e406-4503-beb1-e2b24128c4f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:10.656935 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e876fd-cc02-44b5-abfd-43d6e8bed61b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:10.659049 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22b58c1-e406-4503-beb1-e2b24128c4f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:10.660107 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22b58c1-e406-4503-beb1-e2b24128c4f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:10.661487 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22b58c1-e406-4503-beb1-e2b24128c4f3 admin admin] PATH_INFO: `/v3/projects/ad5f85a315d6404fa21d9bf8aa985c9b` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:10.665285 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22b58c1-e406-4503-beb1-e2b24128c4f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad5f85a315d6404fa21d9bf8aa985c9b)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:10.672347 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22b58c1-e406-4503-beb1-e2b24128c4f3 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:10.731078 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-c9e876fd-cc02-44b5-abfd-43d6e8bed61b None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:19:10.733481 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-c9e876fd-cc02-44b5-abfd-43d6e8bed61b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:10.752267 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 873/1743] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:19:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:19:10.757435 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e876fd-cc02-44b5-abfd-43d6e8bed61b None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:10.776477 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e876fd-cc02-44b5-abfd-43d6e8bed61b None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:10.787966 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb24749-27f9-459c-bb9c-cc0896c7527a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:10.788933 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb24749-27f9-459c-bb9c-cc0896c7527a admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:10.789451 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb24749-27f9-459c-bb9c-cc0896c7527a admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:10.789882 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb24749-27f9-459c-bb9c-cc0896c7527a admin admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:10.795344 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb24749-27f9-459c-bb9c-cc0896c7527a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:19:10.795661 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb24749-27f9-459c-bb9c-cc0896c7527a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:10.797341 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb24749-27f9-459c-bb9c-cc0896c7527a admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:10.851201 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 874/1744] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Aug 25 03:19:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:19:10.852479 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb24749-27f9-459c-bb9c-cc0896c7527a admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:10.868747 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb24749-27f9-459c-bb9c-cc0896c7527a admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:10.924285 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-d22b58c1-e406-4503-beb1-e2b24128c4f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad5f85a315d6404fa21d9bf8aa985c9b', 'request_id': 'req-d22b58c1-e406-4503-beb1-e2b24128c4f3'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:10.999559 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337031db-b67e-444f-a919-dafb7f80542d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:10.999559 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337031db-b67e-444f-a919-dafb7f80542d admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:10.999559 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337031db-b67e-444f-a919-dafb7f80542d admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:10.999559 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337031db-b67e-444f-a919-dafb7f80542d admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:11.002208 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337031db-b67e-444f-a919-dafb7f80542d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:11.002208 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337031db-b67e-444f-a919-dafb7f80542d admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:11.234396 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-d22b58c1-e406-4503-beb1-e2b24128c4f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad5f85a315d6404fa21d9bf8aa985c9b', 'request_id': 'req-d22b58c1-e406-4503-beb1-e2b24128c4f3'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:11.241534 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-d22b58c1-e406-4503-beb1-e2b24128c4f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad5f85a315d6404fa21d9bf8aa985c9b', 'request_id': 'req-d22b58c1-e406-4503-beb1-e2b24128c4f3'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:11.270186 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-d22b58c1-e406-4503-beb1-e2b24128c4f3 admin admin] The token cache is being invalidate because project ad5f85a315d6404fa21d9bf8aa985c9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81995) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:19:11.270799 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-d22b58c1-e406-4503-beb1-e2b24128c4f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:11.271643 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-d22b58c1-e406-4503-beb1-e2b24128c4f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:11.276618 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 871/1745] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:19:10 2022] DELETE /identity/v3/projects/ad5f85a315d6404fa21d9bf8aa985c9b => generated 0 bytes in 712 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:19:11.277729 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22b58c1-e406-4503-beb1-e2b24128c4f3 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:11.282592 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-d22b58c1-e406-4503-beb1-e2b24128c4f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:11.289079 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 875/1746] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:19:10 2022] POST /identity/v3/projects => generated 385 bytes in 437 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:19:11.300466 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337031db-b67e-444f-a919-dafb7f80542d admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:11.308620 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-337031db-b67e-444f-a919-dafb7f80542d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:11.352572 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22b58c1-e406-4503-beb1-e2b24128c4f3 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:11.545185 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337031db-b67e-444f-a919-dafb7f80542d admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:11.548047 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdbc33a-ddba-4f2c-9f7c-cc317623d170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:11.548827 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdbc33a-ddba-4f2c-9f7c-cc317623d170 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:11.549210 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdbc33a-ddba-4f2c-9f7c-cc317623d170 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:11.549557 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdbc33a-ddba-4f2c-9f7c-cc317623d170 admin admin] PATH_INFO: `/v3/users/2e2cb997c7f047328ec6e7837f67d29f` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:11.551233 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdbc33a-ddba-4f2c-9f7c-cc317623d170 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e2cb997c7f047328ec6e7837f67d29f)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:11.556363 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdbc33a-ddba-4f2c-9f7c-cc317623d170 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:11.559183 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb242ff-9e49-4e6a-a0c9-520d44a39ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:11.560915 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb242ff-9e49-4e6a-a0c9-520d44a39ffe admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:11.561393 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb242ff-9e49-4e6a-a0c9-520d44a39ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:11.561868 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb242ff-9e49-4e6a-a0c9-520d44a39ffe admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:11.562758 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb242ff-9e49-4e6a-a0c9-520d44a39ffe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:11.564971 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb242ff-9e49-4e6a-a0c9-520d44a39ffe admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:11.676901 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 876/1747] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:19:11 2022] POST /identity/v3/users => generated 293 bytes in 377 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:19:11.682069 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb242ff-9e49-4e6a-a0c9-520d44a39ffe admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:11.717240 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb242ff-9e49-4e6a-a0c9-520d44a39ffe admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:11.723559 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9f22ee-d60b-4685-83d6-5298819b5372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:11.724266 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9f22ee-d60b-4685-83d6-5298819b5372 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:11.724720 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9f22ee-d60b-4685-83d6-5298819b5372 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:11.725145 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9f22ee-d60b-4685-83d6-5298819b5372 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:11.725945 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9f22ee-d60b-4685-83d6-5298819b5372 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:11.728337 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9f22ee-d60b-4685-83d6-5298819b5372 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:11.763326 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 877/1748] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:19:11 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:19:11.775156 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9f22ee-d60b-4685-83d6-5298819b5372 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:11.788336 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9f22ee-d60b-4685-83d6-5298819b5372 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:11.795451 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b330102-ac48-47a9-b866-a7770fb6d91f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:11.796572 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b330102-ac48-47a9-b866-a7770fb6d91f admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:11.797002 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b330102-ac48-47a9-b866-a7770fb6d91f admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:11.797435 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b330102-ac48-47a9-b866-a7770fb6d91f admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:11.798365 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b330102-ac48-47a9-b866-a7770fb6d91f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:11.803366 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b330102-ac48-47a9-b866-a7770fb6d91f admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:11.848417 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 878/1749] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:19:11 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:19:11.852050 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b330102-ac48-47a9-b866-a7770fb6d91f admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:11.868706 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-dfdbc33a-ddba-4f2c-9f7c-cc317623d170 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e2cb997c7f047328ec6e7837f67d29f'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:11.888632 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b330102-ac48-47a9-b866-a7770fb6d91f admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:11.898261 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044a5918-09b8-4fab-a67e-845715ff205b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:11.899249 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044a5918-09b8-4fab-a67e-845715ff205b admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:11.899740 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044a5918-09b8-4fab-a67e-845715ff205b admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:11.900167 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044a5918-09b8-4fab-a67e-845715ff205b admin admin] PATH_INFO: `/v3/projects/57d35f8cb2a547b7975fe6b490f4705b/users/bbfaf1e55b3f4145949f6d3fb2aee836/roles/3255f77192524a16bc405173106871fc` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:11.917207 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:19:11.922494 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044a5918-09b8-4fab-a67e-845715ff205b admin admin] RBAC: Authorizing `identity:create_grant(project_id=57d35f8cb2a547b7975fe6b490f4705b, user_id=bbfaf1e55b3f4145949f6d3fb2aee836, role_id=3255f77192524a16bc405173106871fc)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:11.925489 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044a5918-09b8-4fab-a67e-845715ff205b admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:11.961052 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-dfdbc33a-ddba-4f2c-9f7c-cc317623d170 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e2cb997c7f047328ec6e7837f67d29f'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:11.961700 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-dfdbc33a-ddba-4f2c-9f7c-cc317623d170 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e2cb997c7f047328ec6e7837f67d29f'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:11.962750 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-dfdbc33a-ddba-4f2c-9f7c-cc317623d170 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e2cb997c7f047328ec6e7837f67d29f'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:11.992935 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 879/1750] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:19:11 2022] PUT /identity/v3/projects/57d35f8cb2a547b7975fe6b490f4705b/users/bbfaf1e55b3f4145949f6d3fb2aee836/roles/3255f77192524a16bc405173106871fc => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:19:11.995702 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-dfdbc33a-ddba-4f2c-9f7c-cc317623d170 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e2cb997c7f047328ec6e7837f67d29f'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:12.008467 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349a2a49-f09a-49d2-ba85-c1defa70547e None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:12.008935 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349a2a49-f09a-49d2-ba85-c1defa70547e None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:12.009423 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349a2a49-f09a-49d2-ba85-c1defa70547e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:12.010850 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:19:12.023771 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 872/1751] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:19:11 2022] DELETE /identity/v3/users/2e2cb997c7f047328ec6e7837f67d29f => generated 0 bytes in 746 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:19:12.052969 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-349a2a49-f09a-49d2-ba85-c1defa70547e None None] MFA Rules not processed for user `bbfaf1e55b3f4145949f6d3fb2aee836`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:19:12.054913 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-349a2a49-f09a-49d2-ba85-c1defa70547e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:12.167499 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 880/1752] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:19:12 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:19:12.168218 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdbc33a-ddba-4f2c-9f7c-cc317623d170 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:12.169951 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-dfdbc33a-ddba-4f2c-9f7c-cc317623d170 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:12.280233 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdbc33a-ddba-4f2c-9f7c-cc317623d170 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:12.288217 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea97546-508b-4099-88f4-ab6689f3f6f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:12.289450 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea97546-508b-4099-88f4-ab6689f3f6f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:12.289794 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea97546-508b-4099-88f4-ab6689f3f6f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:12.290077 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea97546-508b-4099-88f4-ab6689f3f6f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:12.290752 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea97546-508b-4099-88f4-ab6689f3f6f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:12.298318 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea97546-508b-4099-88f4-ab6689f3f6f2 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:12.374124 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349a2a49-f09a-49d2-ba85-c1defa70547e None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:12.375069 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 873/1753] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:19:12 2022] POST /identity/v3/projects => generated 387 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:19:12.375988 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-349a2a49-f09a-49d2-ba85-c1defa70547e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:12.380802 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea97546-508b-4099-88f4-ab6689f3f6f2 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:12.401218 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349a2a49-f09a-49d2-ba85-c1defa70547e None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:12.473865 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea97546-508b-4099-88f4-ab6689f3f6f2 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:12.476294 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b674b7db-a66a-421f-946a-c2cde8f31d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:12.476939 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b674b7db-a66a-421f-946a-c2cde8f31d27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:12.477225 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b674b7db-a66a-421f-946a-c2cde8f31d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:12.477653 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b674b7db-a66a-421f-946a-c2cde8f31d27 admin admin] PATH_INFO: `/v3/projects/b31b2419e7d84fd68089a257a1baf479` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:12.478850 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b674b7db-a66a-421f-946a-c2cde8f31d27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b31b2419e7d84fd68089a257a1baf479)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:12.481025 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b674b7db-a66a-421f-946a-c2cde8f31d27 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:12.483022 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a374a688-8b77-4483-9ac0-ab7ffc125c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:12.483648 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a374a688-8b77-4483-9ac0-ab7ffc125c4e admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:12.483931 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a374a688-8b77-4483-9ac0-ab7ffc125c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:12.484206 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a374a688-8b77-4483-9ac0-ab7ffc125c4e admin admin] PATH_INFO: `/v3/users` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:12.484862 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a374a688-8b77-4483-9ac0-ab7ffc125c4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:12.486832 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a374a688-8b77-4483-9ac0-ab7ffc125c4e admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:12.577907 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 874/1754] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:19:12 2022] POST /identity/v3/users => generated 300 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:19:12.581964 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a374a688-8b77-4483-9ac0-ab7ffc125c4e admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:12.592127 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a374a688-8b77-4483-9ac0-ab7ffc125c4e admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:12.602145 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb92bfa6-569a-4665-bbea-8c9f2d09866a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:12.603510 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb92bfa6-569a-4665-bbea-8c9f2d09866a admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:12.603787 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb92bfa6-569a-4665-bbea-8c9f2d09866a admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:12.604060 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb92bfa6-569a-4665-bbea-8c9f2d09866a admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:12.604755 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb92bfa6-569a-4665-bbea-8c9f2d09866a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:12.606924 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb92bfa6-569a-4665-bbea-8c9f2d09866a admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:12.620128 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 875/1755] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:19:12 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 25 03:19:12.624802 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb92bfa6-569a-4665-bbea-8c9f2d09866a admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:12.624951 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-b674b7db-a66a-421f-946a-c2cde8f31d27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b31b2419e7d84fd68089a257a1baf479', 'request_id': 'req-b674b7db-a66a-421f-946a-c2cde8f31d27'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:12.634488 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb92bfa6-569a-4665-bbea-8c9f2d09866a admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:12.690518 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f847af-516d-40c0-a391-f02b30a9fb05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:12.691184 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f847af-516d-40c0-a391-f02b30a9fb05 admin admin] REQUEST_METHOD: `PUT` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:12.691466 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f847af-516d-40c0-a391-f02b30a9fb05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:12.691746 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f847af-516d-40c0-a391-f02b30a9fb05 admin admin] PATH_INFO: `/v3/projects/4e666053c65849069cf9420e4cbc475b/users/70edd0c2bc084c63b83357a977ad9605/roles/62f19ea272b447ccaab1576cd9bf4fd9` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:12.694731 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:19:12.698954 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-b674b7db-a66a-421f-946a-c2cde8f31d27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b31b2419e7d84fd68089a257a1baf479', 'request_id': 'req-b674b7db-a66a-421f-946a-c2cde8f31d27'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:12.705495 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-b674b7db-a66a-421f-946a-c2cde8f31d27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b31b2419e7d84fd68089a257a1baf479', 'request_id': 'req-b674b7db-a66a-421f-946a-c2cde8f31d27'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:12.713160 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f847af-516d-40c0-a391-f02b30a9fb05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e666053c65849069cf9420e4cbc475b, user_id=70edd0c2bc084c63b83357a977ad9605, role_id=62f19ea272b447ccaab1576cd9bf4fd9)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:12.715885 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f847af-516d-40c0-a391-f02b30a9fb05 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:12.718796 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-b674b7db-a66a-421f-946a-c2cde8f31d27 admin admin] The token cache is being invalidate because project b31b2419e7d84fd68089a257a1baf479 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:19:12.719156 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-b674b7db-a66a-421f-946a-c2cde8f31d27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:12.719749 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-b674b7db-a66a-421f-946a-c2cde8f31d27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:12.721328 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 881/1756] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:19:12 2022] DELETE /identity/v3/projects/b31b2419e7d84fd68089a257a1baf479 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:19:12.726087 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f194469b-1206-4e12-8811-1c56a984ba7c None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:12.726418 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f194469b-1206-4e12-8811-1c56a984ba7c None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:12.726695 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f194469b-1206-4e12-8811-1c56a984ba7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:12.745719 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-f194469b-1206-4e12-8811-1c56a984ba7c None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:19:12.747570 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-f194469b-1206-4e12-8811-1c56a984ba7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:12.755455 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 882/1757] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:19:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:19:12.762788 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d68a846-c181-4aa2-a859-1ba1f5ed3933 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:12.763073 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d68a846-c181-4aa2-a859-1ba1f5ed3933 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:12.763347 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d68a846-c181-4aa2-a859-1ba1f5ed3933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:12.771366 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:19:12.781457 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-7d68a846-c181-4aa2-a859-1ba1f5ed3933 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:19:12.787010 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-7d68a846-c181-4aa2-a859-1ba1f5ed3933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:12.790387 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 883/1758] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:19:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:19:12.794735 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d68a846-c181-4aa2-a859-1ba1f5ed3933 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:12.803287 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d68a846-c181-4aa2-a859-1ba1f5ed3933 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:12.807717 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c811b946-f5dd-4a63-9b2d-926c123c4034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:12.808279 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c811b946-f5dd-4a63-9b2d-926c123c4034 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:12.808559 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c811b946-f5dd-4a63-9b2d-926c123c4034 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:12.808830 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c811b946-f5dd-4a63-9b2d-926c123c4034 admin admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:12.810036 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c811b946-f5dd-4a63-9b2d-926c123c4034 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:19:12.810365 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c811b946-f5dd-4a63-9b2d-926c123c4034 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:12.812018 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c811b946-f5dd-4a63-9b2d-926c123c4034 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:12.827209 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 884/1759] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Aug 25 03:19:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:19:12.827331 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 876/1760] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:19:12 2022] PUT /identity/v3/projects/4e666053c65849069cf9420e4cbc475b/users/70edd0c2bc084c63b83357a977ad9605/roles/62f19ea272b447ccaab1576cd9bf4fd9 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:19:12.830082 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c811b946-f5dd-4a63-9b2d-926c123c4034 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:12.830204 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f847af-516d-40c0-a391-f02b30a9fb05 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:12.832038 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-c811b946-f5dd-4a63-9b2d-926c123c4034 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:12.845399 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c811b946-f5dd-4a63-9b2d-926c123c4034 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:12.896847 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f847af-516d-40c0-a391-f02b30a9fb05 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:12.901928 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b6e1f5-ae6a-4edc-a300-bad184a79154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:12.902615 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b6e1f5-ae6a-4edc-a300-bad184a79154 admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:12.902901 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b6e1f5-ae6a-4edc-a300-bad184a79154 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:12.903696 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b6e1f5-ae6a-4edc-a300-bad184a79154 admin admin] PATH_INFO: `/v3/projects` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:12.904328 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b6e1f5-ae6a-4edc-a300-bad184a79154 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:12.904766 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ca8fb4-046a-4e2e-80e5-c0ca15d15175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:12.905388 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ca8fb4-046a-4e2e-80e5-c0ca15d15175 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:12.905669 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ca8fb4-046a-4e2e-80e5-c0ca15d15175 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:12.905944 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ca8fb4-046a-4e2e-80e5-c0ca15d15175 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:12.906241 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b6e1f5-ae6a-4edc-a300-bad184a79154 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:12.906628 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ca8fb4-046a-4e2e-80e5-c0ca15d15175 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:12.908214 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ca8fb4-046a-4e2e-80e5-c0ca15d15175 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:12.919702 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 885/1761] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:19:12 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:19:12.923783 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ca8fb4-046a-4e2e-80e5-c0ca15d15175 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:12.933278 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ca8fb4-046a-4e2e-80e5-c0ca15d15175 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:12.938408 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f12daf-5f80-47a9-9763-cf2460f3c2c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:12.939069 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f12daf-5f80-47a9-9763-cf2460f3c2c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:12.951055 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f12daf-5f80-47a9-9763-cf2460f3c2c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:12.951055 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f12daf-5f80-47a9-9763-cf2460f3c2c4 admin admin] PATH_INFO: `/v3/projects/4e666053c65849069cf9420e4cbc475b/users/70edd0c2bc084c63b83357a977ad9605/roles/62f19ea272b447ccaab1576cd9bf4fd9` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:12.951055 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f12daf-5f80-47a9-9763-cf2460f3c2c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e666053c65849069cf9420e4cbc475b, user_id=70edd0c2bc084c63b83357a977ad9605, role_id=62f19ea272b447ccaab1576cd9bf4fd9)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:12.951055 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f12daf-5f80-47a9-9763-cf2460f3c2c4 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:12.956837 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 886/1762] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:19:12 2022] PUT /identity/v3/projects/4e666053c65849069cf9420e4cbc475b/users/70edd0c2bc084c63b83357a977ad9605/roles/62f19ea272b447ccaab1576cd9bf4fd9 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:19:12.967418 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d13378-880b-4396-a951-8693ff32efce None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:12.967700 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d13378-880b-4396-a951-8693ff32efce None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:12.967973 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d13378-880b-4396-a951-8693ff32efce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:12.995065 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-d0d13378-880b-4396-a951-8693ff32efce None None] MFA Rules not processed for user `70edd0c2bc084c63b83357a977ad9605`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:19:12.996192 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-d0d13378-880b-4396-a951-8693ff32efce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:13.060882 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 887/1763] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:19:12 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:19:13.070579 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 877/1764] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:19:12 2022] POST /identity/v3/projects => generated 371 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:19:13.072389 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d13378-880b-4396-a951-8693ff32efce None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:13.074120 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-d0d13378-880b-4396-a951-8693ff32efce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:13.076141 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b6e1f5-ae6a-4edc-a300-bad184a79154 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:13.149851 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d13378-880b-4396-a951-8693ff32efce None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:13.157008 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b6e1f5-ae6a-4edc-a300-bad184a79154 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:13.162995 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4fd5f4-8893-4034-b85f-fe4f9f8bf749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:13.163676 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4fd5f4-8893-4034-b85f-fe4f9f8bf749 admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:13.163966 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4fd5f4-8893-4034-b85f-fe4f9f8bf749 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:13.164243 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4fd5f4-8893-4034-b85f-fe4f9f8bf749 admin admin] PATH_INFO: `/v3/users` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:13.165113 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4fd5f4-8893-4034-b85f-fe4f9f8bf749 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:13.167169 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4fd5f4-8893-4034-b85f-fe4f9f8bf749 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:13.174848 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:19:13.217482 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99861a5-5283-44c3-8d5c-6bd7915781f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60193ba9e1c14df3bc9eebcc6f28f2d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:13.218105 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99861a5-5283-44c3-8d5c-6bd7915781f2 service cinder] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:13.218587 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99861a5-5283-44c3-8d5c-6bd7915781f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:13.218847 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99861a5-5283-44c3-8d5c-6bd7915781f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:13.281010 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 878/1765] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:19:13 2022] POST /identity/v3/users => generated 286 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:19:13.283069 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4fd5f4-8893-4034-b85f-fe4f9f8bf749 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:13.284644 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99861a5-5283-44c3-8d5c-6bd7915781f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:13.287393 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99861a5-5283-44c3-8d5c-6bd7915781f2 service cinder] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:13.293292 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4fd5f4-8893-4034-b85f-fe4f9f8bf749 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:13.314895 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 888/1766] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:19:13 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:19:13.317509 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a0387f-38a5-4837-8084-4165857de19d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:13.318462 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a0387f-38a5-4837-8084-4165857de19d admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:13.318767 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a0387f-38a5-4837-8084-4165857de19d admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:13.319015 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a0387f-38a5-4837-8084-4165857de19d admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:13.320044 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a0387f-38a5-4837-8084-4165857de19d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:13.322271 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a0387f-38a5-4837-8084-4165857de19d admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:13.328409 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99861a5-5283-44c3-8d5c-6bd7915781f2 service cinder] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:13.353304 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 879/1767] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:19:13 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:19:13.355204 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99861a5-5283-44c3-8d5c-6bd7915781f2 service cinder] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:13.356543 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a0387f-38a5-4837-8084-4165857de19d admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:13.362313 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfaa6ca-5880-4766-8132-141743a52353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:13.362313 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfaa6ca-5880-4766-8132-141743a52353 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:13.362723 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfaa6ca-5880-4766-8132-141743a52353 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:13.363005 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfaa6ca-5880-4766-8132-141743a52353 admin admin] PATH_INFO: `/v3/users/bbfaf1e55b3f4145949f6d3fb2aee836` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:13.364181 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfaa6ca-5880-4766-8132-141743a52353 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbfaf1e55b3f4145949f6d3fb2aee836)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:13.366230 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfaa6ca-5880-4766-8132-141743a52353 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:13.396848 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a0387f-38a5-4837-8084-4165857de19d admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:13.412115 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07ee07f-a542-47e3-afeb-3369aae5d1b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:13.413381 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07ee07f-a542-47e3-afeb-3369aae5d1b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:13.413771 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07ee07f-a542-47e3-afeb-3369aae5d1b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:13.414054 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07ee07f-a542-47e3-afeb-3369aae5d1b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:13.414774 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07ee07f-a542-47e3-afeb-3369aae5d1b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:13.416401 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07ee07f-a542-47e3-afeb-3369aae5d1b0 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:13.443327 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 880/1768] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:19:13 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:19:13.447756 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07ee07f-a542-47e3-afeb-3369aae5d1b0 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:13.468662 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07ee07f-a542-47e3-afeb-3369aae5d1b0 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:13.478075 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822ef4f8-63e9-4966-a972-430245963155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:13.479115 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822ef4f8-63e9-4966-a972-430245963155 admin admin] REQUEST_METHOD: `PUT` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:13.479680 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822ef4f8-63e9-4966-a972-430245963155 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:13.480170 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822ef4f8-63e9-4966-a972-430245963155 admin admin] PATH_INFO: `/v3/projects/2ee064d3e1a24e238761701a200c70e3/users/b261202b862544599b4dfa8753b98335/roles/3255f77192524a16bc405173106871fc` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:13.508395 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822ef4f8-63e9-4966-a972-430245963155 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ee064d3e1a24e238761701a200c70e3, user_id=b261202b862544599b4dfa8753b98335, role_id=3255f77192524a16bc405173106871fc)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:13.514577 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822ef4f8-63e9-4966-a972-430245963155 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:13.597564 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 881/1769] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:19:13 2022] PUT /identity/v3/projects/2ee064d3e1a24e238761701a200c70e3/users/b261202b862544599b4dfa8753b98335/roles/3255f77192524a16bc405173106871fc => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:19:13.610374 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a877da-e50d-41f2-ae37-9e7d0992aafc None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:13.610878 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a877da-e50d-41f2-ae37-9e7d0992aafc None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:13.611109 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a877da-e50d-41f2-ae37-9e7d0992aafc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:13.665819 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-ecfaa6ca-5880-4766-8132-141743a52353 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbfaf1e55b3f4145949f6d3fb2aee836'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:13.674309 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:19:13.685212 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-24a877da-e50d-41f2-ae37-9e7d0992aafc None None] MFA Rules not processed for user `b261202b862544599b4dfa8753b98335`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:19:13.686868 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-24a877da-e50d-41f2-ae37-9e7d0992aafc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:13.737981 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-ecfaa6ca-5880-4766-8132-141743a52353 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbfaf1e55b3f4145949f6d3fb2aee836'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:13.738642 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-ecfaa6ca-5880-4766-8132-141743a52353 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbfaf1e55b3f4145949f6d3fb2aee836'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:13.746566 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-ecfaa6ca-5880-4766-8132-141743a52353 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbfaf1e55b3f4145949f6d3fb2aee836'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:13.770556 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 882/1770] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:19:13 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:19:13.772059 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-ecfaa6ca-5880-4766-8132-141743a52353 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbfaf1e55b3f4145949f6d3fb2aee836'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:13.774891 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a877da-e50d-41f2-ae37-9e7d0992aafc None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:13.780797 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 889/1771] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:19:13 2022] DELETE /identity/v3/users/bbfaf1e55b3f4145949f6d3fb2aee836 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:19:13.785457 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ab28d2-2d62-4e90-8061-5e47eaa01ac8 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:13.785733 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ab28d2-2d62-4e90-8061-5e47eaa01ac8 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:13.786010 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ab28d2-2d62-4e90-8061-5e47eaa01ac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:13.813229 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-11ab28d2-2d62-4e90-8061-5e47eaa01ac8 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:19:13.814768 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-11ab28d2-2d62-4e90-8061-5e47eaa01ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:13.844179 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:19:13.859174 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a877da-e50d-41f2-ae37-9e7d0992aafc None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:13.889380 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 890/1772] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:19:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:19:13.899432 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ab28d2-2d62-4e90-8061-5e47eaa01ac8 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:13.902595 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-11ab28d2-2d62-4e90-8061-5e47eaa01ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:13.937854 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5075e42-9d23-436b-8851-25e57eaa56f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:13.938513 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5075e42-9d23-436b-8851-25e57eaa56f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:13.938933 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5075e42-9d23-436b-8851-25e57eaa56f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:13.939074 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5075e42-9d23-436b-8851-25e57eaa56f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:13.939684 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5075e42-9d23-436b-8851-25e57eaa56f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:13.941612 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5075e42-9d23-436b-8851-25e57eaa56f0 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:13.983597 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ab28d2-2d62-4e90-8061-5e47eaa01ac8 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:13.988818 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0020cd-ddf4-4870-8398-ee6e976f082a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:13.989458 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0020cd-ddf4-4870-8398-ee6e976f082a service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:13.989736 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0020cd-ddf4-4870-8398-ee6e976f082a service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:13.990010 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0020cd-ddf4-4870-8398-ee6e976f082a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:14.000679 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0020cd-ddf4-4870-8398-ee6e976f082a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:14.002113 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0020cd-ddf4-4870-8398-ee6e976f082a service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:14.013525 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 891/1773] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:19:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 25 03:19:14.050339 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 883/1774] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:19:13 2022] POST /identity/v3/projects => generated 371 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:19:14.054298 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0020cd-ddf4-4870-8398-ee6e976f082a service neutron] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:14.055739 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-6d0020cd-ddf4-4870-8398-ee6e976f082a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:14.072874 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0020cd-ddf4-4870-8398-ee6e976f082a service neutron] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:14.139912 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04468453-d384-4c35-ab98-0cbcd08cbca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:14.140638 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04468453-d384-4c35-ab98-0cbcd08cbca6 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:14.140984 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04468453-d384-4c35-ab98-0cbcd08cbca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:14.141347 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04468453-d384-4c35-ab98-0cbcd08cbca6 admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:14.142103 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04468453-d384-4c35-ab98-0cbcd08cbca6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:14.144076 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04468453-d384-4c35-ab98-0cbcd08cbca6 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:14.210895 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 892/1775] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:19:14 2022] POST /identity/v3/users => generated 292 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:19:14.212926 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5075e42-9d23-436b-8851-25e57eaa56f0 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:14.225543 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5075e42-9d23-436b-8851-25e57eaa56f0 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:14.238610 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb9a295-088d-4752-b663-01ed58074353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:14.239253 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb9a295-088d-4752-b663-01ed58074353 admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:14.239793 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb9a295-088d-4752-b663-01ed58074353 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:14.240080 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb9a295-088d-4752-b663-01ed58074353 admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:14.241343 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb9a295-088d-4752-b663-01ed58074353 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:14.242957 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb9a295-088d-4752-b663-01ed58074353 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:14.256572 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 884/1776] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:19:14 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:19:14.261094 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04468453-d384-4c35-ab98-0cbcd08cbca6 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:14.272121 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04468453-d384-4c35-ab98-0cbcd08cbca6 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:14.276752 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62404e8b-5b65-4295-8699-8f2470817d60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:14.277413 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62404e8b-5b65-4295-8699-8f2470817d60 admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:14.277693 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62404e8b-5b65-4295-8699-8f2470817d60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:14.277963 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62404e8b-5b65-4295-8699-8f2470817d60 admin admin] PATH_INFO: `/v3/projects/63324a77bd4d41409502be3a41fee3c2/users/84cb260697c14d8798769501a6d7b748/roles/62f19ea272b447ccaab1576cd9bf4fd9` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:14.292821 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62404e8b-5b65-4295-8699-8f2470817d60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63324a77bd4d41409502be3a41fee3c2, user_id=84cb260697c14d8798769501a6d7b748, role_id=62f19ea272b447ccaab1576cd9bf4fd9)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:14.296265 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62404e8b-5b65-4295-8699-8f2470817d60 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:14.375445 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 893/1777] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:19:14 2022] PUT /identity/v3/projects/63324a77bd4d41409502be3a41fee3c2/users/84cb260697c14d8798769501a6d7b748/roles/62f19ea272b447ccaab1576cd9bf4fd9 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:19:14.379426 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb9a295-088d-4752-b663-01ed58074353 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:14.389598 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb9a295-088d-4752-b663-01ed58074353 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:14.408095 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62404e8b-5b65-4295-8699-8f2470817d60 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:14.409997 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-62404e8b-5b65-4295-8699-8f2470817d60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:14.473070 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:19:14.499012 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:19:14.535369 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1efa91f-b977-4733-9f7f-b0d1d6d15207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:14.536162 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1efa91f-b977-4733-9f7f-b0d1d6d15207 admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:14.536535 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1efa91f-b977-4733-9f7f-b0d1d6d15207 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:14.536908 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1efa91f-b977-4733-9f7f-b0d1d6d15207 admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:14.538022 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1efa91f-b977-4733-9f7f-b0d1d6d15207 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:14.539977 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1efa91f-b977-4733-9f7f-b0d1d6d15207 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:14.541821 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62404e8b-5b65-4295-8699-8f2470817d60 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:14.556187 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 885/1778] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:19:14 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:19:14.559907 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b161c5b2-5105-4756-b4d0-e1f6a6e9f644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:14.560045 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1efa91f-b977-4733-9f7f-b0d1d6d15207 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:14.560572 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b161c5b2-5105-4756-b4d0-e1f6a6e9f644 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:14.562527 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b161c5b2-5105-4756-b4d0-e1f6a6e9f644 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:14.562527 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b161c5b2-5105-4756-b4d0-e1f6a6e9f644 admin admin] PATH_INFO: `/v3/projects/57d35f8cb2a547b7975fe6b490f4705b` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:14.562848 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b161c5b2-5105-4756-b4d0-e1f6a6e9f644 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57d35f8cb2a547b7975fe6b490f4705b)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:14.565663 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b161c5b2-5105-4756-b4d0-e1f6a6e9f644 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:14.571189 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1efa91f-b977-4733-9f7f-b0d1d6d15207 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:14.576703 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e90883a-6283-4722-adc6-4021d4ffe94d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:14.577420 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e90883a-6283-4722-adc6-4021d4ffe94d admin admin] REQUEST_METHOD: `PUT` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:14.577779 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e90883a-6283-4722-adc6-4021d4ffe94d admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:14.578129 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e90883a-6283-4722-adc6-4021d4ffe94d admin admin] PATH_INFO: `/v3/projects/63324a77bd4d41409502be3a41fee3c2/users/84cb260697c14d8798769501a6d7b748/roles/62f19ea272b447ccaab1576cd9bf4fd9` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:14.580243 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e90883a-6283-4722-adc6-4021d4ffe94d admin admin] RBAC: Authorizing `identity:create_grant(project_id=63324a77bd4d41409502be3a41fee3c2, user_id=84cb260697c14d8798769501a6d7b748, role_id=62f19ea272b447ccaab1576cd9bf4fd9)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:14.583169 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e90883a-6283-4722-adc6-4021d4ffe94d admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:14.594956 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 886/1779] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:19:14 2022] PUT /identity/v3/projects/63324a77bd4d41409502be3a41fee3c2/users/84cb260697c14d8798769501a6d7b748/roles/62f19ea272b447ccaab1576cd9bf4fd9 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:19:14.608322 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c698e44e-131f-4a85-a7f1-f414fe29f228 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:14.608611 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c698e44e-131f-4a85-a7f1-f414fe29f228 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:14.608896 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c698e44e-131f-4a85-a7f1-f414fe29f228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:14.642326 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-c698e44e-131f-4a85-a7f1-f414fe29f228 None None] MFA Rules not processed for user `84cb260697c14d8798769501a6d7b748`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:19:14.643198 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-c698e44e-131f-4a85-a7f1-f414fe29f228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:14.687508 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-b161c5b2-5105-4756-b4d0-e1f6a6e9f644 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57d35f8cb2a547b7975fe6b490f4705b', 'request_id': 'req-b161c5b2-5105-4756-b4d0-e1f6a6e9f644'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:14.734731 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 887/1780] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:19:14 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:19:14.746258 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c698e44e-131f-4a85-a7f1-f414fe29f228 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:14.748247 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-c698e44e-131f-4a85-a7f1-f414fe29f228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:14.766127 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-b161c5b2-5105-4756-b4d0-e1f6a6e9f644 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57d35f8cb2a547b7975fe6b490f4705b', 'request_id': 'req-b161c5b2-5105-4756-b4d0-e1f6a6e9f644'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:14.772823 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-b161c5b2-5105-4756-b4d0-e1f6a6e9f644 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57d35f8cb2a547b7975fe6b490f4705b', 'request_id': 'req-b161c5b2-5105-4756-b4d0-e1f6a6e9f644'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:14.784480 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-b161c5b2-5105-4756-b4d0-e1f6a6e9f644 admin admin] The token cache is being invalidate because project 57d35f8cb2a547b7975fe6b490f4705b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:19:14.784862 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-b161c5b2-5105-4756-b4d0-e1f6a6e9f644 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:14.785592 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-b161c5b2-5105-4756-b4d0-e1f6a6e9f644 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:14.787436 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 894/1781] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:19:14 2022] DELETE /identity/v3/projects/57d35f8cb2a547b7975fe6b490f4705b => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:19:14.791054 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b161c5b2-5105-4756-b4d0-e1f6a6e9f644 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:14.792421 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-b161c5b2-5105-4756-b4d0-e1f6a6e9f644 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:14.810048 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c698e44e-131f-4a85-a7f1-f414fe29f228 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:14.853316 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:19:14.858377 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5369ad39-cbdc-4204-85cb-b4aa3ebeed6b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60193ba9e1c14df3bc9eebcc6f28f2d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:14.858997 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5369ad39-cbdc-4204-85cb-b4aa3ebeed6b service cinder] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:14.859275 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5369ad39-cbdc-4204-85cb-b4aa3ebeed6b service cinder] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:14.859367 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b161c5b2-5105-4756-b4d0-e1f6a6e9f644 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:14.859550 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5369ad39-cbdc-4204-85cb-b4aa3ebeed6b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:14.861260 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-5369ad39-cbdc-4204-85cb-b4aa3ebeed6b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:14.864261 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695c0ca2-f748-45d9-82ce-5628fb9cab75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:14.864907 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695c0ca2-f748-45d9-82ce-5628fb9cab75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:14.865189 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695c0ca2-f748-45d9-82ce-5628fb9cab75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:14.869783 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695c0ca2-f748-45d9-82ce-5628fb9cab75 admin admin] PATH_INFO: `/v3/users/70edd0c2bc084c63b83357a977ad9605` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:14.869783 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695c0ca2-f748-45d9-82ce-5628fb9cab75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70edd0c2bc084c63b83357a977ad9605)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:14.869783 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695c0ca2-f748-45d9-82ce-5628fb9cab75 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:14.925097 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5369ad39-cbdc-4204-85cb-b4aa3ebeed6b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:14.926566 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5369ad39-cbdc-4204-85cb-b4aa3ebeed6b service cinder] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:14.936511 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 888/1782] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:19:14 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 25 03:19:15.038099 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-695c0ca2-f748-45d9-82ce-5628fb9cab75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70edd0c2bc084c63b83357a977ad9605'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:15.114734 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-695c0ca2-f748-45d9-82ce-5628fb9cab75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70edd0c2bc084c63b83357a977ad9605'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:15.115444 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-695c0ca2-f748-45d9-82ce-5628fb9cab75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70edd0c2bc084c63b83357a977ad9605'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:15.125569 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-695c0ca2-f748-45d9-82ce-5628fb9cab75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70edd0c2bc084c63b83357a977ad9605'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:15.133170 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:19:15.138272 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-695c0ca2-f748-45d9-82ce-5628fb9cab75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70edd0c2bc084c63b83357a977ad9605'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:15.147620 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 895/1783] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:19:14 2022] DELETE /identity/v3/users/70edd0c2bc084c63b83357a977ad9605 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:19:15.191436 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5369ad39-cbdc-4204-85cb-b4aa3ebeed6b service cinder] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:15.192863 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-5369ad39-cbdc-4204-85cb-b4aa3ebeed6b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:15.205408 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5369ad39-cbdc-4204-85cb-b4aa3ebeed6b service cinder] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:15.279435 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac3a90b-5865-4703-80c8-9c754ba06c7f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60193ba9e1c14df3bc9eebcc6f28f2d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:15.280074 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac3a90b-5865-4703-80c8-9c754ba06c7f service cinder] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:15.280362 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac3a90b-5865-4703-80c8-9c754ba06c7f service cinder] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:15.280637 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac3a90b-5865-4703-80c8-9c754ba06c7f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:15.284833 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-aac3a90b-5865-4703-80c8-9c754ba06c7f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:15.354859 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac3a90b-5865-4703-80c8-9c754ba06c7f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:15.357117 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac3a90b-5865-4703-80c8-9c754ba06c7f service cinder] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:15.368542 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 889/1784] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:19:15 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:19:15.809959 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695c0ca2-f748-45d9-82ce-5628fb9cab75 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:15.812398 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-695c0ca2-f748-45d9-82ce-5628fb9cab75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:15.989560 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695c0ca2-f748-45d9-82ce-5628fb9cab75 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:15.998184 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a15896-341f-4a9a-9442-83d581ed464e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:15.998983 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a15896-341f-4a9a-9442-83d581ed464e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:15.999201 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a15896-341f-4a9a-9442-83d581ed464e admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:15.999475 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a15896-341f-4a9a-9442-83d581ed464e admin admin] PATH_INFO: `/v3/projects/4e666053c65849069cf9420e4cbc475b` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:16.000760 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a15896-341f-4a9a-9442-83d581ed464e admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e666053c65849069cf9420e4cbc475b)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:16.003377 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a15896-341f-4a9a-9442-83d581ed464e admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:16.119676 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-87a15896-341f-4a9a-9442-83d581ed464e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e666053c65849069cf9420e4cbc475b', 'request_id': 'req-87a15896-341f-4a9a-9442-83d581ed464e'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:16.189336 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-87a15896-341f-4a9a-9442-83d581ed464e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e666053c65849069cf9420e4cbc475b', 'request_id': 'req-87a15896-341f-4a9a-9442-83d581ed464e'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:16.196356 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-87a15896-341f-4a9a-9442-83d581ed464e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e666053c65849069cf9420e4cbc475b', 'request_id': 'req-87a15896-341f-4a9a-9442-83d581ed464e'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:16.216786 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-87a15896-341f-4a9a-9442-83d581ed464e admin admin] The token cache is being invalidate because project 4e666053c65849069cf9420e4cbc475b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:19:16.221775 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-87a15896-341f-4a9a-9442-83d581ed464e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:16.221775 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-87a15896-341f-4a9a-9442-83d581ed464e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:16.221775 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 896/1785] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:19:15 2022] DELETE /identity/v3/projects/4e666053c65849069cf9420e4cbc475b => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:19:16.225145 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e63b1b1-8a9b-4f53-8c41-d510f4f5048e None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:16.225654 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e63b1b1-8a9b-4f53-8c41-d510f4f5048e None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:16.225743 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e63b1b1-8a9b-4f53-8c41-d510f4f5048e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:16.249398 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-2e63b1b1-8a9b-4f53-8c41-d510f4f5048e None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:19:16.251099 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-2e63b1b1-8a9b-4f53-8c41-d510f4f5048e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:16.321067 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:19:16.335100 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 890/1786] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:19:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:19:16.349454 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8360381a-6253-4dcd-9edc-33bfdab51a42 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:16.349729 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8360381a-6253-4dcd-9edc-33bfdab51a42 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:16.350026 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8360381a-6253-4dcd-9edc-33bfdab51a42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:16.371829 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-8360381a-6253-4dcd-9edc-33bfdab51a42 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:19:16.373689 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-8360381a-6253-4dcd-9edc-33bfdab51a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:16.389011 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 897/1787] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:19:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:19:16.393431 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e63b1b1-8a9b-4f53-8c41-d510f4f5048e None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:16.410170 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e63b1b1-8a9b-4f53-8c41-d510f4f5048e None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:16.416297 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5285dc4-8586-4db6-b890-ee1a25c45da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:16.417030 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5285dc4-8586-4db6-b890-ee1a25c45da7 admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:16.417509 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5285dc4-8586-4db6-b890-ee1a25c45da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:16.417861 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5285dc4-8586-4db6-b890-ee1a25c45da7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:16.418666 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5285dc4-8586-4db6-b890-ee1a25c45da7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:19:16.419226 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5285dc4-8586-4db6-b890-ee1a25c45da7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:16.420971 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5285dc4-8586-4db6-b890-ee1a25c45da7 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:16.434746 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 891/1788] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Aug 25 03:19:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:19:16.439608 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8360381a-6253-4dcd-9edc-33bfdab51a42 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:16.449102 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8360381a-6253-4dcd-9edc-33bfdab51a42 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:16.454369 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfc2335-eb41-4acf-9236-0d05b31673d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:16.454943 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfc2335-eb41-4acf-9236-0d05b31673d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:16.455222 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfc2335-eb41-4acf-9236-0d05b31673d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:16.455492 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfc2335-eb41-4acf-9236-0d05b31673d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:16.456173 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfc2335-eb41-4acf-9236-0d05b31673d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:16.458068 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfc2335-eb41-4acf-9236-0d05b31673d7 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:16.529766 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 898/1789] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:19:16 2022] POST /identity/v3/projects => generated 383 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:19:16.534367 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5285dc4-8586-4db6-b890-ee1a25c45da7 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:16.546774 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5285dc4-8586-4db6-b890-ee1a25c45da7 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:16.605401 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cbd72f-26f1-4759-b2a4-c099beb9fb4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:16.606021 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cbd72f-26f1-4759-b2a4-c099beb9fb4b admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:16.606381 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cbd72f-26f1-4759-b2a4-c099beb9fb4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:16.606659 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cbd72f-26f1-4759-b2a4-c099beb9fb4b admin admin] PATH_INFO: `/v3/users` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:16.607299 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cbd72f-26f1-4759-b2a4-c099beb9fb4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:16.609262 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cbd72f-26f1-4759-b2a4-c099beb9fb4b admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:16.724786 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 892/1790] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:19:16 2022] POST /identity/v3/users => generated 292 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:19:16.732575 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfc2335-eb41-4acf-9236-0d05b31673d7 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:16.744699 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfc2335-eb41-4acf-9236-0d05b31673d7 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:16.749957 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e5b6f7-9991-4275-a1fe-b16e3647b06a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:16.750611 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e5b6f7-9991-4275-a1fe-b16e3647b06a admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:16.750893 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e5b6f7-9991-4275-a1fe-b16e3647b06a admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:16.751180 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e5b6f7-9991-4275-a1fe-b16e3647b06a admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:16.751816 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e5b6f7-9991-4275-a1fe-b16e3647b06a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:16.753459 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e5b6f7-9991-4275-a1fe-b16e3647b06a admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:16.766768 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 899/1791] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:19:16 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:19:16.770691 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cbd72f-26f1-4759-b2a4-c099beb9fb4b admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:16.780974 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cbd72f-26f1-4759-b2a4-c099beb9fb4b admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:16.787480 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20f39dc-8a90-499f-a561-59e6b484165b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:16.788093 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20f39dc-8a90-499f-a561-59e6b484165b admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:16.788372 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20f39dc-8a90-499f-a561-59e6b484165b admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:16.788643 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20f39dc-8a90-499f-a561-59e6b484165b admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:16.789295 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20f39dc-8a90-499f-a561-59e6b484165b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:16.791165 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20f39dc-8a90-499f-a561-59e6b484165b admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:16.806602 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 893/1792] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:19:16 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:19:16.811239 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e5b6f7-9991-4275-a1fe-b16e3647b06a admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:16.821960 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e5b6f7-9991-4275-a1fe-b16e3647b06a admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:16.827889 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbb1ab2-1014-4780-a560-7ee610ef79a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:16.828672 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbb1ab2-1014-4780-a560-7ee610ef79a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:16.829199 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbb1ab2-1014-4780-a560-7ee610ef79a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:16.829574 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbb1ab2-1014-4780-a560-7ee610ef79a2 admin admin] PATH_INFO: `/v3/projects/17ec8c3525614c4bba01263430d6c32b/users/752712f3bbab451fa13b419bcf4f3f71/roles/3255f77192524a16bc405173106871fc` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:16.845613 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbb1ab2-1014-4780-a560-7ee610ef79a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17ec8c3525614c4bba01263430d6c32b, user_id=752712f3bbab451fa13b419bcf4f3f71, role_id=3255f77192524a16bc405173106871fc)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:16.848608 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbb1ab2-1014-4780-a560-7ee610ef79a2 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:16.932646 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 900/1793] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:19:16 2022] PUT /identity/v3/projects/17ec8c3525614c4bba01263430d6c32b/users/752712f3bbab451fa13b419bcf4f3f71/roles/3255f77192524a16bc405173106871fc => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:19:16.946883 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a409c15-7908-4be9-b087-54d7a351e59c None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:16.947401 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a409c15-7908-4be9-b087-54d7a351e59c None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:16.947748 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a409c15-7908-4be9-b087-54d7a351e59c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:16.986067 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-3a409c15-7908-4be9-b087-54d7a351e59c None None] MFA Rules not processed for user `752712f3bbab451fa13b419bcf4f3f71`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:19:16.988054 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-3a409c15-7908-4be9-b087-54d7a351e59c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:17.060596 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 894/1794] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:19:16 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:19:17.065954 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbb1ab2-1014-4780-a560-7ee610ef79a2 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:17.084251 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=3 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:19:17.127649 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbb1ab2-1014-4780-a560-7ee610ef79a2 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:17.132262 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf9acd0-274a-4338-a24e-3db3fd6488d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:17.132872 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf9acd0-274a-4338-a24e-3db3fd6488d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:17.133156 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf9acd0-274a-4338-a24e-3db3fd6488d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:17.133431 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf9acd0-274a-4338-a24e-3db3fd6488d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:17.134050 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf9acd0-274a-4338-a24e-3db3fd6488d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:17.135996 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf9acd0-274a-4338-a24e-3db3fd6488d6 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:17.230507 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 901/1795] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:19:17 2022] POST /identity/v3/projects => generated 387 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:19:17.235240 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a409c15-7908-4be9-b087-54d7a351e59c None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:17.246498 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a409c15-7908-4be9-b087-54d7a351e59c None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:17.291063 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:19:17.302920 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0542e9-4c6f-4cb4-9cb3-af8804983e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:17.303541 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0542e9-4c6f-4cb4-9cb3-af8804983e2c admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:17.303826 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0542e9-4c6f-4cb4-9cb3-af8804983e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:17.304103 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0542e9-4c6f-4cb4-9cb3-af8804983e2c admin admin] PATH_INFO: `/v3/users` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:17.304837 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0542e9-4c6f-4cb4-9cb3-af8804983e2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:17.306794 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0542e9-4c6f-4cb4-9cb3-af8804983e2c admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:17.377618 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 895/1796] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:19:17 2022] POST /identity/v3/users => generated 294 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:19:17.381631 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf9acd0-274a-4338-a24e-3db3fd6488d6 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:17.391905 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf9acd0-274a-4338-a24e-3db3fd6488d6 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:17.396480 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c28936c-da0a-4f26-b1c3-58a1eefa80dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:17.397085 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c28936c-da0a-4f26-b1c3-58a1eefa80dc admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:17.397363 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c28936c-da0a-4f26-b1c3-58a1eefa80dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:17.397636 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c28936c-da0a-4f26-b1c3-58a1eefa80dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:17.398333 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c28936c-da0a-4f26-b1c3-58a1eefa80dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:17.399892 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c28936c-da0a-4f26-b1c3-58a1eefa80dc admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:17.410933 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 902/1797] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:19:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:19:17.414913 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0542e9-4c6f-4cb4-9cb3-af8804983e2c admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:17.425539 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0542e9-4c6f-4cb4-9cb3-af8804983e2c admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:17.430243 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7d8e80-c042-4d69-aec8-4a9b8ce51c04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:17.430886 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7d8e80-c042-4d69-aec8-4a9b8ce51c04 admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:17.431168 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7d8e80-c042-4d69-aec8-4a9b8ce51c04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:17.431445 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7d8e80-c042-4d69-aec8-4a9b8ce51c04 admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:17.432098 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7d8e80-c042-4d69-aec8-4a9b8ce51c04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:17.433817 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7d8e80-c042-4d69-aec8-4a9b8ce51c04 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:17.443616 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 896/1798] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:19:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:19:17.447655 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c28936c-da0a-4f26-b1c3-58a1eefa80dc admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:17.457610 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c28936c-da0a-4f26-b1c3-58a1eefa80dc admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:17.462119 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda9ff69-d0a0-4a69-b562-92615a24d8c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:17.462799 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda9ff69-d0a0-4a69-b562-92615a24d8c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:17.463079 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda9ff69-d0a0-4a69-b562-92615a24d8c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:17.463353 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda9ff69-d0a0-4a69-b562-92615a24d8c6 admin admin] PATH_INFO: `/v3/projects/feaf957becd94a3ea8544c906b37ddfc/users/57aa1499b90c48fda1c38c96f69a755c/roles/3255f77192524a16bc405173106871fc` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:17.478049 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda9ff69-d0a0-4a69-b562-92615a24d8c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=feaf957becd94a3ea8544c906b37ddfc, user_id=57aa1499b90c48fda1c38c96f69a755c, role_id=3255f77192524a16bc405173106871fc)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:17.480661 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda9ff69-d0a0-4a69-b562-92615a24d8c6 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:17.546842 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 903/1799] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:19:17 2022] PUT /identity/v3/projects/feaf957becd94a3ea8544c906b37ddfc/users/57aa1499b90c48fda1c38c96f69a755c/roles/3255f77192524a16bc405173106871fc => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:19:17.557000 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69c9f9e-4502-4ad8-a06f-8c0a8dabff75 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:17.557292 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69c9f9e-4502-4ad8-a06f-8c0a8dabff75 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:17.557575 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69c9f9e-4502-4ad8-a06f-8c0a8dabff75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:17.586512 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-e69c9f9e-4502-4ad8-a06f-8c0a8dabff75 None None] MFA Rules not processed for user `57aa1499b90c48fda1c38c96f69a755c`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:19:17.587956 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-e69c9f9e-4502-4ad8-a06f-8c0a8dabff75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:17.654213 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 897/1800] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:19:17 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:19:17.658771 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda9ff69-d0a0-4a69-b562-92615a24d8c6 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:17.718726 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda9ff69-d0a0-4a69-b562-92615a24d8c6 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:17.723402 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c3d01f-8b5b-4a2a-a45e-6dbc2ec8988e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:17.724003 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c3d01f-8b5b-4a2a-a45e-6dbc2ec8988e admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:17.724287 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c3d01f-8b5b-4a2a-a45e-6dbc2ec8988e admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:17.724564 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c3d01f-8b5b-4a2a-a45e-6dbc2ec8988e admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:17.725176 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c3d01f-8b5b-4a2a-a45e-6dbc2ec8988e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:17.727136 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c3d01f-8b5b-4a2a-a45e-6dbc2ec8988e admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:17.819749 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 904/1801] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:19:17 2022] POST /identity/v3/projects => generated 385 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:19:17.824067 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69c9f9e-4502-4ad8-a06f-8c0a8dabff75 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:17.835477 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69c9f9e-4502-4ad8-a06f-8c0a8dabff75 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:17.895538 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501e1db6-9077-418c-a972-7fe218f99ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:17.896166 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501e1db6-9077-418c-a972-7fe218f99ee3 admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:17.896453 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501e1db6-9077-418c-a972-7fe218f99ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:17.896728 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501e1db6-9077-418c-a972-7fe218f99ee3 admin admin] PATH_INFO: `/v3/users` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:17.897425 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501e1db6-9077-418c-a972-7fe218f99ee3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:17.899458 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501e1db6-9077-418c-a972-7fe218f99ee3 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:17.999956 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 898/1802] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:19:17 2022] POST /identity/v3/users => generated 299 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:19:18.000969 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c3d01f-8b5b-4a2a-a45e-6dbc2ec8988e admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:18.015510 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c3d01f-8b5b-4a2a-a45e-6dbc2ec8988e admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:18.021435 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf55884b-1273-4b75-8c35-8bc6ac03d8ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:18.022357 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf55884b-1273-4b75-8c35-8bc6ac03d8ef admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:18.023089 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf55884b-1273-4b75-8c35-8bc6ac03d8ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:18.023668 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf55884b-1273-4b75-8c35-8bc6ac03d8ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:18.024511 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf55884b-1273-4b75-8c35-8bc6ac03d8ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:18.026472 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf55884b-1273-4b75-8c35-8bc6ac03d8ef admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:18.044392 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501e1db6-9077-418c-a972-7fe218f99ee3 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:18.051873 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 905/1803] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:19:17 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 25 03:19:18.055688 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501e1db6-9077-418c-a972-7fe218f99ee3 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:18.068385 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5027d151-cc8a-46e3-b1b9-71f0414bd085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:18.068385 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5027d151-cc8a-46e3-b1b9-71f0414bd085 admin admin] REQUEST_METHOD: `PUT` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:18.068385 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5027d151-cc8a-46e3-b1b9-71f0414bd085 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:18.068385 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5027d151-cc8a-46e3-b1b9-71f0414bd085 admin admin] PATH_INFO: `/v3/projects/2e0db11694ce4317881f103ab8a659ad/users/030de938bb44423f9da013f081022a90/roles/62f19ea272b447ccaab1576cd9bf4fd9` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:18.113275 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5027d151-cc8a-46e3-b1b9-71f0414bd085 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e0db11694ce4317881f103ab8a659ad, user_id=030de938bb44423f9da013f081022a90, role_id=62f19ea272b447ccaab1576cd9bf4fd9)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:18.116114 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5027d151-cc8a-46e3-b1b9-71f0414bd085 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:18.226804 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 899/1804] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:19:18 2022] PUT /identity/v3/projects/2e0db11694ce4317881f103ab8a659ad/users/030de938bb44423f9da013f081022a90/roles/62f19ea272b447ccaab1576cd9bf4fd9 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:19:18.230652 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf55884b-1273-4b75-8c35-8bc6ac03d8ef admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:18.241995 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf55884b-1273-4b75-8c35-8bc6ac03d8ef admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:18.284863 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:19:18.304773 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aabf621-150b-4aa3-8916-e01fd41d71ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:18.305533 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aabf621-150b-4aa3-8916-e01fd41d71ed admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:18.305907 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aabf621-150b-4aa3-8916-e01fd41d71ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:18.306186 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aabf621-150b-4aa3-8916-e01fd41d71ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:18.306883 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aabf621-150b-4aa3-8916-e01fd41d71ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:18.309811 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aabf621-150b-4aa3-8916-e01fd41d71ed admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:18.326817 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 906/1805] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:19:18 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:19:18.331360 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5027d151-cc8a-46e3-b1b9-71f0414bd085 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:18.342428 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5027d151-cc8a-46e3-b1b9-71f0414bd085 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:18.350463 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef7504a-3ffc-42f9-b83c-ef7c3f41a3cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:18.351153 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef7504a-3ffc-42f9-b83c-ef7c3f41a3cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:18.351436 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef7504a-3ffc-42f9-b83c-ef7c3f41a3cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:18.351716 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef7504a-3ffc-42f9-b83c-ef7c3f41a3cd admin admin] PATH_INFO: `/v3/projects/2e0db11694ce4317881f103ab8a659ad/users/030de938bb44423f9da013f081022a90/roles/62f19ea272b447ccaab1576cd9bf4fd9` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:18.353666 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef7504a-3ffc-42f9-b83c-ef7c3f41a3cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e0db11694ce4317881f103ab8a659ad, user_id=030de938bb44423f9da013f081022a90, role_id=62f19ea272b447ccaab1576cd9bf4fd9)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:18.356703 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef7504a-3ffc-42f9-b83c-ef7c3f41a3cd admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:18.368171 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 900/1806] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:19:18 2022] PUT /identity/v3/projects/2e0db11694ce4317881f103ab8a659ad/users/030de938bb44423f9da013f081022a90/roles/62f19ea272b447ccaab1576cd9bf4fd9 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:19:18.379188 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b506f9-849a-4113-8368-1c6aef4a5542 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:18.379478 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b506f9-849a-4113-8368-1c6aef4a5542 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:18.379756 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b506f9-849a-4113-8368-1c6aef4a5542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:18.411500 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-53b506f9-849a-4113-8368-1c6aef4a5542 None None] MFA Rules not processed for user `030de938bb44423f9da013f081022a90`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:19:18.412984 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-53b506f9-849a-4113-8368-1c6aef4a5542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:18.488724 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 907/1807] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:19:18 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:19:18.502299 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef7504a-3ffc-42f9-b83c-ef7c3f41a3cd admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:18.504422 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-6ef7504a-3ffc-42f9-b83c-ef7c3f41a3cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:18.523669 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:19:18.593771 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef7504a-3ffc-42f9-b83c-ef7c3f41a3cd admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:18.598139 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d44e8c-c717-430a-ac4a-8838912bb306 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60193ba9e1c14df3bc9eebcc6f28f2d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:18.598806 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d44e8c-c717-430a-ac4a-8838912bb306 service cinder] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:18.599091 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d44e8c-c717-430a-ac4a-8838912bb306 service cinder] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:18.599364 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d44e8c-c717-430a-ac4a-8838912bb306 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:18.610936 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d44e8c-c717-430a-ac4a-8838912bb306 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:18.612717 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d44e8c-c717-430a-ac4a-8838912bb306 service cinder] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:18.625577 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 901/1808] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:19:18 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:19:19.697051 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b506f9-849a-4113-8368-1c6aef4a5542 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:19.708622 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b506f9-849a-4113-8368-1c6aef4a5542 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:19.712570 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdfb6e0-f351-44ec-8dc2-0904dbbc1d35 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60193ba9e1c14df3bc9eebcc6f28f2d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:19.713124 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdfb6e0-f351-44ec-8dc2-0904dbbc1d35 service cinder] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:19.713407 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdfb6e0-f351-44ec-8dc2-0904dbbc1d35 service cinder] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:19.713681 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdfb6e0-f351-44ec-8dc2-0904dbbc1d35 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:19.776348 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdfb6e0-f351-44ec-8dc2-0904dbbc1d35 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:19.777665 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdfb6e0-f351-44ec-8dc2-0904dbbc1d35 service cinder] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:19.786932 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 908/1809] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:19:19 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:19:23.075125 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d44e8c-c717-430a-ac4a-8838912bb306 service cinder] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:23.084108 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d44e8c-c717-430a-ac4a-8838912bb306 service cinder] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:23.089031 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53867faa-3478-4836-b54e-828ffe64bc31 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60193ba9e1c14df3bc9eebcc6f28f2d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:23.089031 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53867faa-3478-4836-b54e-828ffe64bc31 service cinder] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:23.089031 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53867faa-3478-4836-b54e-828ffe64bc31 service cinder] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:23.089031 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53867faa-3478-4836-b54e-828ffe64bc31 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:23.090842 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-53867faa-3478-4836-b54e-828ffe64bc31 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:23.198023 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53867faa-3478-4836-b54e-828ffe64bc31 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:23.200403 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53867faa-3478-4836-b54e-828ffe64bc31 service cinder] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:23.215985 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 902/1810] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:19:23 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:19:29.388325 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7b83d5-893c-4987-a60d-d0c4c9cce52f None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:29.388325 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7b83d5-893c-4987-a60d-d0c4c9cce52f None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:29.388325 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7b83d5-893c-4987-a60d-d0c4c9cce52f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:29.388325 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-dd7b83d5-893c-4987-a60d-d0c4c9cce52f None None] MFA Rules not processed for user `a0a1a7f091774842907fd08de15e8058`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:19:29.388325 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-dd7b83d5-893c-4987-a60d-d0c4c9cce52f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:29.414403 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 909/1811] 10.222.0.17 () {58 vars in 1015 bytes} [Thu Aug 25 03:19:29 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:19:29.431452 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53867faa-3478-4836-b54e-828ffe64bc31 service cinder] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:29.432797 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7b83d5-893c-4987-a60d-d0c4c9cce52f None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:29.433020 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-53867faa-3478-4836-b54e-828ffe64bc31 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:29.434240 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-dd7b83d5-893c-4987-a60d-d0c4c9cce52f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:29.444450 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7b83d5-893c-4987-a60d-d0c4c9cce52f None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:29.444707 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53867faa-3478-4836-b54e-828ffe64bc31 service cinder] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:29.448081 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5d25a4-085b-4a19-aeb9-63c657da49d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:29.448299 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863d7695-8ccb-4e1b-82ea-359b2237191e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:29.448943 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5d25a4-085b-4a19-aeb9-63c657da49d7 service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:29.449154 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863d7695-8ccb-4e1b-82ea-359b2237191e service neutron] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:29.449267 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5d25a4-085b-4a19-aeb9-63c657da49d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:29.449438 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863d7695-8ccb-4e1b-82ea-359b2237191e service neutron] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:29.449545 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5d25a4-085b-4a19-aeb9-63c657da49d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:29.449715 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863d7695-8ccb-4e1b-82ea-359b2237191e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:29.454474 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:19:29.456874 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5d25a4-085b-4a19-aeb9-63c657da49d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:29.457094 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863d7695-8ccb-4e1b-82ea-359b2237191e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:29.458387 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5d25a4-085b-4a19-aeb9-63c657da49d7 service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:29.458727 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863d7695-8ccb-4e1b-82ea-359b2237191e service neutron] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:29.466545 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 910/1812] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:19:29 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:19:29.467401 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 903/1813] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:19:29 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 25 03:19:30.554117 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc692f35-dea3-4602-b7f2-94654f074c74 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:30.556494 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc692f35-dea3-4602-b7f2-94654f074c74 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:30.556494 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc692f35-dea3-4602-b7f2-94654f074c74 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:30.559111 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 911/1814] 10.222.0.58 () {54 vars in 921 bytes} [Thu Aug 25 03:19:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:19:30.559877 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a3b108-f2e7-4228-ae9d-a1068d59cb1b None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:30.560340 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a3b108-f2e7-4228-ae9d-a1068d59cb1b None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:30.560431 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a3b108-f2e7-4228-ae9d-a1068d59cb1b None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:30.562009 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 904/1815] 10.222.0.58 () {54 vars in 921 bytes} [Thu Aug 25 03:19:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:19:30.568008 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d592f693-544f-48a3-85b4-9c565be0b63d None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:30.568857 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d592f693-544f-48a3-85b4-9c565be0b63d None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:30.569557 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d592f693-544f-48a3-85b4-9c565be0b63d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:30.584961 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beab850-1682-4cda-a9a2-b186d0fdbf5b None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:30.589281 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beab850-1682-4cda-a9a2-b186d0fdbf5b None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:30.589281 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beab850-1682-4cda-a9a2-b186d0fdbf5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:30.613598 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-d592f693-544f-48a3-85b4-9c565be0b63d None None] MFA Rules not processed for user `0cd171bf4df94f8fab397e42835feeab`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:19:30.614100 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-9beab850-1682-4cda-a9a2-b186d0fdbf5b None None] MFA Rules not processed for user `0cd171bf4df94f8fab397e42835feeab`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:19:30.615504 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-9beab850-1682-4cda-a9a2-b186d0fdbf5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:30.615819 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-d592f693-544f-48a3-85b4-9c565be0b63d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:30.748934 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 912/1816] 10.222.0.58 () {58 vars in 1018 bytes} [Thu Aug 25 03:19:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:19:30.748934 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d592f693-544f-48a3-85b4-9c565be0b63d None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:30.749489 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 905/1817] 10.222.0.58 () {58 vars in 1018 bytes} [Thu Aug 25 03:19:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:19:30.749590 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-d592f693-544f-48a3-85b4-9c565be0b63d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:30.749590 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d592f693-544f-48a3-85b4-9c565be0b63d None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:30.749807 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beab850-1682-4cda-a9a2-b186d0fdbf5b None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:30.749807 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-9beab850-1682-4cda-a9a2-b186d0fdbf5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:30.749971 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7222bbfd-97e8-4b5f-b201-7d32bce7aaee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cd171bf4df94f8fab397e42835feeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:30.749971 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7222bbfd-97e8-4b5f-b201-7d32bce7aaee service nova] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:30.749971 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7222bbfd-97e8-4b5f-b201-7d32bce7aaee service nova] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:30.749971 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7222bbfd-97e8-4b5f-b201-7d32bce7aaee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:30.750428 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beab850-1682-4cda-a9a2-b186d0fdbf5b None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:30.750535 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7222bbfd-97e8-4b5f-b201-7d32bce7aaee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:30.752260 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7222bbfd-97e8-4b5f-b201-7d32bce7aaee service nova] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:30.756152 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1270ca9f-fc5b-4808-8739-1bd1ddc4d9f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cd171bf4df94f8fab397e42835feeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:30.756758 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1270ca9f-fc5b-4808-8739-1bd1ddc4d9f8 service nova] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:30.757035 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1270ca9f-fc5b-4808-8739-1bd1ddc4d9f8 service nova] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:30.757313 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1270ca9f-fc5b-4808-8739-1bd1ddc4d9f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:30.763286 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 913/1818] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:19:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:19:30.767393 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1270ca9f-fc5b-4808-8739-1bd1ddc4d9f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:30.774464 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1270ca9f-fc5b-4808-8739-1bd1ddc4d9f8 service nova] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:30.779867 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 906/1819] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:19:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:19:36.490115 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7222bbfd-97e8-4b5f-b201-7d32bce7aaee service nova] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:36.495916 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-7222bbfd-97e8-4b5f-b201-7d32bce7aaee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:36.656541 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7222bbfd-97e8-4b5f-b201-7d32bce7aaee service nova] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:36.661626 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cff84f-3a37-4d29-9658-fcd0d92125a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:36.662315 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cff84f-3a37-4d29-9658-fcd0d92125a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:36.662604 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cff84f-3a37-4d29-9658-fcd0d92125a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:36.662929 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cff84f-3a37-4d29-9658-fcd0d92125a0 admin admin] PATH_INFO: `/v3/users/539fbb2860474b69883aa3d5e88bd4f0` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:36.664043 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cff84f-3a37-4d29-9658-fcd0d92125a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=539fbb2860474b69883aa3d5e88bd4f0)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:36.666012 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cff84f-3a37-4d29-9658-fcd0d92125a0 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:36.843487 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-b5cff84f-3a37-4d29-9658-fcd0d92125a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '539fbb2860474b69883aa3d5e88bd4f0'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:36.932819 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-b5cff84f-3a37-4d29-9658-fcd0d92125a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '539fbb2860474b69883aa3d5e88bd4f0'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:36.933839 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-b5cff84f-3a37-4d29-9658-fcd0d92125a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '539fbb2860474b69883aa3d5e88bd4f0'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:36.946715 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-b5cff84f-3a37-4d29-9658-fcd0d92125a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '539fbb2860474b69883aa3d5e88bd4f0'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:36.962584 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:19:36.965439 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-b5cff84f-3a37-4d29-9658-fcd0d92125a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '539fbb2860474b69883aa3d5e88bd4f0'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:36.972594 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 914/1820] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:19:36 2022] DELETE /identity/v3/users/539fbb2860474b69883aa3d5e88bd4f0 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:19:36.978561 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24897e0b-8be4-4e42-b6fa-14b911b1ba8a None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:36.979059 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24897e0b-8be4-4e42-b6fa-14b911b1ba8a None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:36.979170 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24897e0b-8be4-4e42-b6fa-14b911b1ba8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:36.999156 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-24897e0b-8be4-4e42-b6fa-14b911b1ba8a None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:19:37.000587 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-24897e0b-8be4-4e42-b6fa-14b911b1ba8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:37.010991 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:19:37.064497 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 907/1821] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:19:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:19:37.074826 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cff84f-3a37-4d29-9658-fcd0d92125a0 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:37.078580 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-b5cff84f-3a37-4d29-9658-fcd0d92125a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:37.155243 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cff84f-3a37-4d29-9658-fcd0d92125a0 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:37.161313 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d477e119-d184-47bc-a824-16c43c1ab3c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:37.162041 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d477e119-d184-47bc-a824-16c43c1ab3c7 service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:37.162560 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d477e119-d184-47bc-a824-16c43c1ab3c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:37.162811 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d477e119-d184-47bc-a824-16c43c1ab3c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:37.176420 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d477e119-d184-47bc-a824-16c43c1ab3c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:37.177934 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d477e119-d184-47bc-a824-16c43c1ab3c7 service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:37.200455 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 915/1822] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:19:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:19:37.718791 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24897e0b-8be4-4e42-b6fa-14b911b1ba8a None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:37.718791 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-24897e0b-8be4-4e42-b6fa-14b911b1ba8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:37.727557 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24897e0b-8be4-4e42-b6fa-14b911b1ba8a None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:37.748493 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ab697f-ef62-4df4-850c-dc2dc1350140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:37.749163 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ab697f-ef62-4df4-850c-dc2dc1350140 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:37.749474 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ab697f-ef62-4df4-850c-dc2dc1350140 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:37.749751 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ab697f-ef62-4df4-850c-dc2dc1350140 admin admin] PATH_INFO: `/v3/projects/48be9cd602794803911476ce3390dd55` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:37.751082 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ab697f-ef62-4df4-850c-dc2dc1350140 admin admin] RBAC: Authorizing `identity:delete_project(project_id=48be9cd602794803911476ce3390dd55)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:37.753167 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ab697f-ef62-4df4-850c-dc2dc1350140 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:37.846115 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d477e119-d184-47bc-a824-16c43c1ab3c7 service neutron] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:37.849564 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-d477e119-d184-47bc-a824-16c43c1ab3c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:37.863520 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d477e119-d184-47bc-a824-16c43c1ab3c7 service neutron] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:37.866552 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-c6ab697f-ef62-4df4-850c-dc2dc1350140 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48be9cd602794803911476ce3390dd55', 'request_id': 'req-c6ab697f-ef62-4df4-850c-dc2dc1350140'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:37.932565 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8edeaf-df38-4a62-a306-5d6698c31e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:37.933513 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8edeaf-df38-4a62-a306-5d6698c31e73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:37.933933 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8edeaf-df38-4a62-a306-5d6698c31e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:37.934381 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8edeaf-df38-4a62-a306-5d6698c31e73 admin admin] PATH_INFO: `/v3/users/5f5c083feda3427496276f825d078d20` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:37.936179 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8edeaf-df38-4a62-a306-5d6698c31e73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f5c083feda3427496276f825d078d20)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:37.939497 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8edeaf-df38-4a62-a306-5d6698c31e73 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:37.966655 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-c6ab697f-ef62-4df4-850c-dc2dc1350140 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48be9cd602794803911476ce3390dd55', 'request_id': 'req-c6ab697f-ef62-4df4-850c-dc2dc1350140'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:37.975452 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-c6ab697f-ef62-4df4-850c-dc2dc1350140 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48be9cd602794803911476ce3390dd55', 'request_id': 'req-c6ab697f-ef62-4df4-850c-dc2dc1350140'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:37.993772 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-c6ab697f-ef62-4df4-850c-dc2dc1350140 admin admin] The token cache is being invalidate because project 48be9cd602794803911476ce3390dd55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81995) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:19:37.994319 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-c6ab697f-ef62-4df4-850c-dc2dc1350140 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:37.995474 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-c6ab697f-ef62-4df4-850c-dc2dc1350140 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:38.000279 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 908/1823] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:19:37 2022] DELETE /identity/v3/projects/48be9cd602794803911476ce3390dd55 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:19:38.003225 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0225f294-2dc1-428a-a41a-a0c73d8cbc69 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:38.003512 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0225f294-2dc1-428a-a41a-a0c73d8cbc69 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:38.003999 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0225f294-2dc1-428a-a41a-a0c73d8cbc69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:38.033306 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-0225f294-2dc1-428a-a41a-a0c73d8cbc69 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:19:38.035139 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-0225f294-2dc1-428a-a41a-a0c73d8cbc69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:38.044216 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 909/1824] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:19:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:19:38.055664 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e90a32-dab9-4a79-bf6d-0bd6433e68c1 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:38.056183 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e90a32-dab9-4a79-bf6d-0bd6433e68c1 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:38.063175 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e90a32-dab9-4a79-bf6d-0bd6433e68c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:38.077215 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-d9e90a32-dab9-4a79-bf6d-0bd6433e68c1 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:19:38.078459 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-d9e90a32-dab9-4a79-bf6d-0bd6433e68c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:38.087526 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 910/1825] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:19:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:19:38.091872 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e90a32-dab9-4a79-bf6d-0bd6433e68c1 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:38.104979 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e90a32-dab9-4a79-bf6d-0bd6433e68c1 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:38.110121 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f50b50-3473-47ac-b440-9a49ba5042bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:38.110892 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f50b50-3473-47ac-b440-9a49ba5042bb admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:38.111041 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f50b50-3473-47ac-b440-9a49ba5042bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:38.111544 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f50b50-3473-47ac-b440-9a49ba5042bb admin admin] PATH_INFO: `/v3/domains` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:38.112199 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f50b50-3473-47ac-b440-9a49ba5042bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:19:38.112501 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f50b50-3473-47ac-b440-9a49ba5042bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:38.114514 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f50b50-3473-47ac-b440-9a49ba5042bb admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:38.142622 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 911/1826] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Aug 25 03:19:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:19:38.146738 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f50b50-3473-47ac-b440-9a49ba5042bb admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:38.158532 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f50b50-3473-47ac-b440-9a49ba5042bb admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:38.163514 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e00aa1-2cca-41b3-8bbd-63eff01ac7a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:38.164246 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e00aa1-2cca-41b3-8bbd-63eff01ac7a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:38.164530 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e00aa1-2cca-41b3-8bbd-63eff01ac7a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:38.164802 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e00aa1-2cca-41b3-8bbd-63eff01ac7a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:38.165424 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e00aa1-2cca-41b3-8bbd-63eff01ac7a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:38.167603 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e00aa1-2cca-41b3-8bbd-63eff01ac7a4 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:38.214385 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-4e8edeaf-df38-4a62-a306-5d6698c31e73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f5c083feda3427496276f825d078d20'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:38.292982 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-4e8edeaf-df38-4a62-a306-5d6698c31e73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f5c083feda3427496276f825d078d20'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:38.294684 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-4e8edeaf-df38-4a62-a306-5d6698c31e73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f5c083feda3427496276f825d078d20'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:38.303266 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-4e8edeaf-df38-4a62-a306-5d6698c31e73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f5c083feda3427496276f825d078d20'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:38.304089 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 912/1827] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:19:38 2022] POST /identity/v3/projects => generated 405 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:19:38.308799 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e00aa1-2cca-41b3-8bbd-63eff01ac7a4 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:38.316972 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-4e8edeaf-df38-4a62-a306-5d6698c31e73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f5c083feda3427496276f825d078d20'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:38.325975 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 916/1828] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:19:37 2022] DELETE /identity/v3/users/5f5c083feda3427496276f825d078d20 => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:19:38.331166 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fa01a3-0e04-4906-af9b-20f275d1106f None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:38.331459 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fa01a3-0e04-4906-af9b-20f275d1106f None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:38.331900 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fa01a3-0e04-4906-af9b-20f275d1106f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:38.354454 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-41fa01a3-0e04-4906-af9b-20f275d1106f None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:19:38.355661 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-41fa01a3-0e04-4906-af9b-20f275d1106f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:38.365208 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:19:38.368596 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:19:38.382733 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e00aa1-2cca-41b3-8bbd-63eff01ac7a4 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:38.415443 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 917/1829] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:19:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:19:38.416412 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fa01a3-0e04-4906-af9b-20f275d1106f None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:38.417724 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-41fa01a3-0e04-4906-af9b-20f275d1106f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:38.441633 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534b1f23-2552-4eb4-9226-24f5ea480b4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:38.442296 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534b1f23-2552-4eb4-9226-24f5ea480b4a admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:38.442633 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534b1f23-2552-4eb4-9226-24f5ea480b4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:38.442863 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534b1f23-2552-4eb4-9226-24f5ea480b4a admin admin] PATH_INFO: `/v3/users` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:38.443508 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534b1f23-2552-4eb4-9226-24f5ea480b4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:38.445585 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534b1f23-2552-4eb4-9226-24f5ea480b4a admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:38.479507 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fa01a3-0e04-4906-af9b-20f275d1106f None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:38.483119 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35188785-dea5-4093-adf4-109ab853fcda service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60193ba9e1c14df3bc9eebcc6f28f2d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:38.483723 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35188785-dea5-4093-adf4-109ab853fcda service cinder] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:38.484005 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35188785-dea5-4093-adf4-109ab853fcda service cinder] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:38.484299 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35188785-dea5-4093-adf4-109ab853fcda service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:38.486135 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-35188785-dea5-4093-adf4-109ab853fcda service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:38.515640 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 913/1830] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:19:38 2022] POST /identity/v3/users => generated 303 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:19:38.517198 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534b1f23-2552-4eb4-9226-24f5ea480b4a admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:38.520756 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-534b1f23-2552-4eb4-9226-24f5ea480b4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:38.545930 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35188785-dea5-4093-adf4-109ab853fcda service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:38.547509 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35188785-dea5-4093-adf4-109ab853fcda service cinder] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:38.557276 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 918/1831] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:19:38 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:19:38.559686 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35188785-dea5-4093-adf4-109ab853fcda service cinder] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:38.559686 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-35188785-dea5-4093-adf4-109ab853fcda service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:38.573710 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35188785-dea5-4093-adf4-109ab853fcda service cinder] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:38.584451 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534b1f23-2552-4eb4-9226-24f5ea480b4a admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:38.588224 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa7f1e1-65a1-45c6-9883-f1a4d944678b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:38.588521 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a839bcbe-5d79-471a-a851-19b2d2cbfa78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:38.589032 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa7f1e1-65a1-45c6-9883-f1a4d944678b admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:38.589154 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a839bcbe-5d79-471a-a851-19b2d2cbfa78 service neutron] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:38.589333 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa7f1e1-65a1-45c6-9883-f1a4d944678b admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:38.589443 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a839bcbe-5d79-471a-a851-19b2d2cbfa78 service neutron] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:38.589594 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa7f1e1-65a1-45c6-9883-f1a4d944678b admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:38.589816 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a839bcbe-5d79-471a-a851-19b2d2cbfa78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:38.590316 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa7f1e1-65a1-45c6-9883-f1a4d944678b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:38.592596 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa7f1e1-65a1-45c6-9883-f1a4d944678b admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:38.602322 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a839bcbe-5d79-471a-a851-19b2d2cbfa78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:38.603690 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 919/1832] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:19:38 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:19:38.603928 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a839bcbe-5d79-471a-a851-19b2d2cbfa78 service neutron] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:38.608141 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa7f1e1-65a1-45c6-9883-f1a4d944678b admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:38.620883 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa7f1e1-65a1-45c6-9883-f1a4d944678b admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:38.623457 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 914/1833] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:19:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:19:38.628371 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0749c275-360b-4eec-aae5-c66de1c73965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:38.629143 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0749c275-360b-4eec-aae5-c66de1c73965 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:38.634530 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0749c275-360b-4eec-aae5-c66de1c73965 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:38.634530 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0749c275-360b-4eec-aae5-c66de1c73965 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:38.634530 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0749c275-360b-4eec-aae5-c66de1c73965 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:38.634530 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0749c275-360b-4eec-aae5-c66de1c73965 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:38.655683 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 920/1834] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:19:38 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:19:38.656876 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a839bcbe-5d79-471a-a851-19b2d2cbfa78 service neutron] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:38.667041 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a839bcbe-5d79-471a-a851-19b2d2cbfa78 service neutron] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:38.672183 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5fdb9e-3a65-465d-bf59-9266b23d44a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:38.673371 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5fdb9e-3a65-465d-bf59-9266b23d44a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:38.673654 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5fdb9e-3a65-465d-bf59-9266b23d44a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:38.673930 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5fdb9e-3a65-465d-bf59-9266b23d44a1 admin admin] PATH_INFO: `/v3/projects/e87afdbba324458198f8e53262bbac65/users/1086ed247eda4b0fbd8e5d36ec88f2d0/roles/3255f77192524a16bc405173106871fc` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:38.694859 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5fdb9e-3a65-465d-bf59-9266b23d44a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e87afdbba324458198f8e53262bbac65, user_id=1086ed247eda4b0fbd8e5d36ec88f2d0, role_id=3255f77192524a16bc405173106871fc)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:38.697482 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5fdb9e-3a65-465d-bf59-9266b23d44a1 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:38.794900 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 915/1835] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:19:38 2022] PUT /identity/v3/projects/e87afdbba324458198f8e53262bbac65/users/1086ed247eda4b0fbd8e5d36ec88f2d0/roles/3255f77192524a16bc405173106871fc => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:19:38.807493 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb11931-2451-4578-b2bd-7149652a0826 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:38.807872 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb11931-2451-4578-b2bd-7149652a0826 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:38.808229 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb11931-2451-4578-b2bd-7149652a0826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:38.838099 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-1cb11931-2451-4578-b2bd-7149652a0826 None None] MFA Rules not processed for user `1086ed247eda4b0fbd8e5d36ec88f2d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:19:38.839420 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-1cb11931-2451-4578-b2bd-7149652a0826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:38.882197 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:19:38.914053 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 921/1836] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:19:38 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:19:38.918893 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5fdb9e-3a65-465d-bf59-9266b23d44a1 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:39.013036 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5fdb9e-3a65-465d-bf59-9266b23d44a1 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:39.032010 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5515ed5d-2c09-46c0-aa26-3aaf28338ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:39.032888 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5515ed5d-2c09-46c0-aa26-3aaf28338ab3 admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:39.034458 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5515ed5d-2c09-46c0-aa26-3aaf28338ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:39.034458 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5515ed5d-2c09-46c0-aa26-3aaf28338ab3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:39.035219 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5515ed5d-2c09-46c0-aa26-3aaf28338ab3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:39.037312 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5515ed5d-2c09-46c0-aa26-3aaf28338ab3 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:39.044355 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:19:39.166747 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 916/1837] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:19:38 2022] POST /identity/v3/projects => generated 405 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:19:39.170912 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb11931-2451-4578-b2bd-7149652a0826 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:39.182553 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb11931-2451-4578-b2bd-7149652a0826 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:39.213467 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5515ed5d-2c09-46c0-aa26-3aaf28338ab3 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:39.214988 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-5515ed5d-2c09-46c0-aa26-3aaf28338ab3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:39.264144 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b78f04a-8776-4de9-be2b-921dbb0c3b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:39.264906 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b78f04a-8776-4de9-be2b-921dbb0c3b20 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:39.265244 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b78f04a-8776-4de9-be2b-921dbb0c3b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:39.265533 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b78f04a-8776-4de9-be2b-921dbb0c3b20 admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:39.266273 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b78f04a-8776-4de9-be2b-921dbb0c3b20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:39.269154 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b78f04a-8776-4de9-be2b-921dbb0c3b20 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:39.295464 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5515ed5d-2c09-46c0-aa26-3aaf28338ab3 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:39.301695 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de77da37-f393-4fcb-b0b2-dadf2fd7096b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:39.305153 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de77da37-f393-4fcb-b0b2-dadf2fd7096b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:39.305153 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de77da37-f393-4fcb-b0b2-dadf2fd7096b admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:39.305153 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de77da37-f393-4fcb-b0b2-dadf2fd7096b admin admin] PATH_INFO: `/v3/projects/9a42b3ecbaf04e558979c9cf1f06e236` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:39.305153 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de77da37-f393-4fcb-b0b2-dadf2fd7096b admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a42b3ecbaf04e558979c9cf1f06e236)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:39.307842 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de77da37-f393-4fcb-b0b2-dadf2fd7096b admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:39.356791 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 922/1838] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:19:39 2022] POST /identity/v3/users => generated 309 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:19:39.361399 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b78f04a-8776-4de9-be2b-921dbb0c3b20 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:39.373293 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b78f04a-8776-4de9-be2b-921dbb0c3b20 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:39.380551 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f3a321-43f6-4156-bf3f-2b3d50108e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:39.380730 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f3a321-43f6-4156-bf3f-2b3d50108e63 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:39.380928 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f3a321-43f6-4156-bf3f-2b3d50108e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:39.381158 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f3a321-43f6-4156-bf3f-2b3d50108e63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:39.381835 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f3a321-43f6-4156-bf3f-2b3d50108e63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:39.383470 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f3a321-43f6-4156-bf3f-2b3d50108e63 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:39.398889 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 923/1839] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:19:39 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 25 03:19:39.403151 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f3a321-43f6-4156-bf3f-2b3d50108e63 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:39.408929 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-de77da37-f393-4fcb-b0b2-dadf2fd7096b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a42b3ecbaf04e558979c9cf1f06e236', 'request_id': 'req-de77da37-f393-4fcb-b0b2-dadf2fd7096b'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:39.414403 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f3a321-43f6-4156-bf3f-2b3d50108e63 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:39.463624 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-de77da37-f393-4fcb-b0b2-dadf2fd7096b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a42b3ecbaf04e558979c9cf1f06e236', 'request_id': 'req-de77da37-f393-4fcb-b0b2-dadf2fd7096b'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:39.470215 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-de77da37-f393-4fcb-b0b2-dadf2fd7096b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a42b3ecbaf04e558979c9cf1f06e236', 'request_id': 'req-de77da37-f393-4fcb-b0b2-dadf2fd7096b'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:39.483012 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-de77da37-f393-4fcb-b0b2-dadf2fd7096b admin admin] The token cache is being invalidate because project 9a42b3ecbaf04e558979c9cf1f06e236 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81995) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:19:39.483300 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-de77da37-f393-4fcb-b0b2-dadf2fd7096b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:39.487485 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-de77da37-f393-4fcb-b0b2-dadf2fd7096b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:39.487609 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2554b4-b3ad-405b-aaff-f69555edca6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:39.487609 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2554b4-b3ad-405b-aaff-f69555edca6e admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:39.487811 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 917/1840] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:19:39 2022] DELETE /identity/v3/projects/9a42b3ecbaf04e558979c9cf1f06e236 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:19:39.487881 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2554b4-b3ad-405b-aaff-f69555edca6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:39.487881 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2554b4-b3ad-405b-aaff-f69555edca6e admin admin] PATH_INFO: `/v3/projects/02586873b0a549f393d8653d449e591f/users/6f9c19326ccf4ecf91ee6730dbfdf7b7/roles/62f19ea272b447ccaab1576cd9bf4fd9` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:39.490227 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de77da37-f393-4fcb-b0b2-dadf2fd7096b admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:39.491799 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-de77da37-f393-4fcb-b0b2-dadf2fd7096b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:39.506179 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de77da37-f393-4fcb-b0b2-dadf2fd7096b admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:39.508182 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2554b4-b3ad-405b-aaff-f69555edca6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=02586873b0a549f393d8653d449e591f, user_id=6f9c19326ccf4ecf91ee6730dbfdf7b7, role_id=62f19ea272b447ccaab1576cd9bf4fd9)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:39.511454 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2554b4-b3ad-405b-aaff-f69555edca6e admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:39.511575 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2548504-7ae8-44db-92bf-b7f2ea877a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:39.512414 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2548504-7ae8-44db-92bf-b7f2ea877a51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:39.516177 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2548504-7ae8-44db-92bf-b7f2ea877a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:39.516177 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2548504-7ae8-44db-92bf-b7f2ea877a51 admin admin] PATH_INFO: `/v3/users/d248205be76440399cfdee05d8e508c2` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:39.516177 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2548504-7ae8-44db-92bf-b7f2ea877a51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d248205be76440399cfdee05d8e508c2)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:39.516556 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2548504-7ae8-44db-92bf-b7f2ea877a51 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:39.578078 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 924/1841] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:19:39 2022] PUT /identity/v3/projects/02586873b0a549f393d8653d449e591f/users/6f9c19326ccf4ecf91ee6730dbfdf7b7/roles/62f19ea272b447ccaab1576cd9bf4fd9 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:19:39.582193 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2554b4-b3ad-405b-aaff-f69555edca6e admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:39.583963 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-4a2554b4-b3ad-405b-aaff-f69555edca6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:39.625652 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:19:39.646676 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2554b4-b3ad-405b-aaff-f69555edca6e admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:39.651314 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe016637-d772-4c5c-bb52-4a23b20e3143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:39.651924 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe016637-d772-4c5c-bb52-4a23b20e3143 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:39.652205 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe016637-d772-4c5c-bb52-4a23b20e3143 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:39.652477 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe016637-d772-4c5c-bb52-4a23b20e3143 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:39.653115 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe016637-d772-4c5c-bb52-4a23b20e3143 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:39.654739 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe016637-d772-4c5c-bb52-4a23b20e3143 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:39.664456 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 925/1842] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:19:39 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 25 03:19:39.668500 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe016637-d772-4c5c-bb52-4a23b20e3143 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:39.678098 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe016637-d772-4c5c-bb52-4a23b20e3143 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:39.682766 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af99ce33-e03b-41f5-a3a2-00102ba64005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:39.686160 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af99ce33-e03b-41f5-a3a2-00102ba64005 admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:39.686160 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af99ce33-e03b-41f5-a3a2-00102ba64005 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:39.686160 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af99ce33-e03b-41f5-a3a2-00102ba64005 admin admin] PATH_INFO: `/v3/projects/02586873b0a549f393d8653d449e591f/users/6f9c19326ccf4ecf91ee6730dbfdf7b7/roles/62f19ea272b447ccaab1576cd9bf4fd9` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:39.686160 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af99ce33-e03b-41f5-a3a2-00102ba64005 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02586873b0a549f393d8653d449e591f, user_id=6f9c19326ccf4ecf91ee6730dbfdf7b7, role_id=62f19ea272b447ccaab1576cd9bf4fd9)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:39.689022 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af99ce33-e03b-41f5-a3a2-00102ba64005 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:39.689623 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-c2548504-7ae8-44db-92bf-b7f2ea877a51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd248205be76440399cfdee05d8e508c2'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:39.700805 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 926/1843] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:19:39 2022] PUT /identity/v3/projects/02586873b0a549f393d8653d449e591f/users/6f9c19326ccf4ecf91ee6730dbfdf7b7/roles/62f19ea272b447ccaab1576cd9bf4fd9 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:19:39.711084 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16595a08-2315-409b-ac8e-4d18b44e2b20 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:39.711368 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16595a08-2315-409b-ac8e-4d18b44e2b20 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:39.711643 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16595a08-2315-409b-ac8e-4d18b44e2b20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:39.737438 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-c2548504-7ae8-44db-92bf-b7f2ea877a51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd248205be76440399cfdee05d8e508c2'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:39.738213 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-c2548504-7ae8-44db-92bf-b7f2ea877a51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd248205be76440399cfdee05d8e508c2'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:39.738968 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-c2548504-7ae8-44db-92bf-b7f2ea877a51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd248205be76440399cfdee05d8e508c2'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:39.744129 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-16595a08-2315-409b-ac8e-4d18b44e2b20 None None] MFA Rules not processed for user `6f9c19326ccf4ecf91ee6730dbfdf7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:19:39.746221 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-16595a08-2315-409b-ac8e-4d18b44e2b20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:39.755500 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-c2548504-7ae8-44db-92bf-b7f2ea877a51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd248205be76440399cfdee05d8e508c2'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:39.756994 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=80,keystone:INSERT=3,keystone:DELETE=17 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:19:39.772816 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 918/1844] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:19:39 2022] DELETE /identity/v3/users/d248205be76440399cfdee05d8e508c2 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:19:39.829116 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 927/1845] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:19:39 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:19:39.840096 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2548504-7ae8-44db-92bf-b7f2ea877a51 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:39.841716 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-c2548504-7ae8-44db-92bf-b7f2ea877a51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:39.900179 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2548504-7ae8-44db-92bf-b7f2ea877a51 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:39.903793 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9bf571-38da-4d4c-b632-2d2874fe6ffd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60193ba9e1c14df3bc9eebcc6f28f2d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:39.904433 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9bf571-38da-4d4c-b632-2d2874fe6ffd service cinder] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:39.904741 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9bf571-38da-4d4c-b632-2d2874fe6ffd service cinder] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:39.905016 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9bf571-38da-4d4c-b632-2d2874fe6ffd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:39.966226 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9bf571-38da-4d4c-b632-2d2874fe6ffd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:39.967702 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9bf571-38da-4d4c-b632-2d2874fe6ffd service cinder] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:39.978338 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 919/1846] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:19:39 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:19:40.260682 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16595a08-2315-409b-ac8e-4d18b44e2b20 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:40.262867 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-16595a08-2315-409b-ac8e-4d18b44e2b20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:40.356189 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16595a08-2315-409b-ac8e-4d18b44e2b20 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:40.409009 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:19:40.418011 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaec71e0-97fd-4d71-90fe-4eda32066a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:40.418804 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaec71e0-97fd-4d71-90fe-4eda32066a3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:40.419085 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaec71e0-97fd-4d71-90fe-4eda32066a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:40.419359 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaec71e0-97fd-4d71-90fe-4eda32066a3a admin admin] PATH_INFO: `/v3/projects/2c8c1fad8d25471986efab70fcaed338` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:40.420585 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaec71e0-97fd-4d71-90fe-4eda32066a3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c8c1fad8d25471986efab70fcaed338)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:40.422652 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaec71e0-97fd-4d71-90fe-4eda32066a3a admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:40.538248 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-aaec71e0-97fd-4d71-90fe-4eda32066a3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c8c1fad8d25471986efab70fcaed338', 'request_id': 'req-aaec71e0-97fd-4d71-90fe-4eda32066a3a'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:40.651824 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-aaec71e0-97fd-4d71-90fe-4eda32066a3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c8c1fad8d25471986efab70fcaed338', 'request_id': 'req-aaec71e0-97fd-4d71-90fe-4eda32066a3a'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:40.657845 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-aaec71e0-97fd-4d71-90fe-4eda32066a3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c8c1fad8d25471986efab70fcaed338', 'request_id': 'req-aaec71e0-97fd-4d71-90fe-4eda32066a3a'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:40.668811 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-aaec71e0-97fd-4d71-90fe-4eda32066a3a admin admin] The token cache is being invalidate because project 2c8c1fad8d25471986efab70fcaed338 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:19:40.669096 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-aaec71e0-97fd-4d71-90fe-4eda32066a3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:40.669728 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-aaec71e0-97fd-4d71-90fe-4eda32066a3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:40.671299 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 928/1847] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:19:40 2022] DELETE /identity/v3/projects/2c8c1fad8d25471986efab70fcaed338 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:19:40.676355 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504db869-6f85-4219-aa25-87fdb26cbee9 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:40.676856 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504db869-6f85-4219-aa25-87fdb26cbee9 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:40.676951 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504db869-6f85-4219-aa25-87fdb26cbee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:40.697865 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-504db869-6f85-4219-aa25-87fdb26cbee9 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:19:40.699385 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-504db869-6f85-4219-aa25-87fdb26cbee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:40.758299 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 920/1848] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:19:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:19:40.768215 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d07c01-e77a-42a7-9401-529d0d2bb572 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:40.768508 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d07c01-e77a-42a7-9401-529d0d2bb572 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:40.768784 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d07c01-e77a-42a7-9401-529d0d2bb572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:40.788040 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-07d07c01-e77a-42a7-9401-529d0d2bb572 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:19:40.789306 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-07d07c01-e77a-42a7-9401-529d0d2bb572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:40.797080 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 929/1849] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:19:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:19:40.801507 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504db869-6f85-4219-aa25-87fdb26cbee9 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:40.811436 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504db869-6f85-4219-aa25-87fdb26cbee9 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:40.816696 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4150c541-45db-4ee2-9ceb-20d009d75a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:40.817303 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4150c541-45db-4ee2-9ceb-20d009d75a29 admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:40.817582 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4150c541-45db-4ee2-9ceb-20d009d75a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:40.817857 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4150c541-45db-4ee2-9ceb-20d009d75a29 admin admin] PATH_INFO: `/v3/domains` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:40.818608 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4150c541-45db-4ee2-9ceb-20d009d75a29 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:19:40.818914 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4150c541-45db-4ee2-9ceb-20d009d75a29 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:40.821276 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4150c541-45db-4ee2-9ceb-20d009d75a29 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:40.833357 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 921/1850] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Aug 25 03:19:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:19:40.837867 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d07c01-e77a-42a7-9401-529d0d2bb572 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:40.847145 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d07c01-e77a-42a7-9401-529d0d2bb572 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:40.852426 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ce18e2-5da3-4a79-9b72-f69603ecf7d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:40.853039 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ce18e2-5da3-4a79-9b72-f69603ecf7d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:40.853320 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ce18e2-5da3-4a79-9b72-f69603ecf7d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:40.853596 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ce18e2-5da3-4a79-9b72-f69603ecf7d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:40.854217 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ce18e2-5da3-4a79-9b72-f69603ecf7d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:40.856156 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ce18e2-5da3-4a79-9b72-f69603ecf7d5 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:40.947740 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 930/1851] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:19:40 2022] POST /identity/v3/projects => generated 365 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:19:40.953402 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4150c541-45db-4ee2-9ceb-20d009d75a29 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:40.963885 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4150c541-45db-4ee2-9ceb-20d009d75a29 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:40.988990 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:19:41.021477 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aecf2e8-58e2-43ff-bbbc-fe7d48e9c15e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:41.022091 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aecf2e8-58e2-43ff-bbbc-fe7d48e9c15e admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:41.022411 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aecf2e8-58e2-43ff-bbbc-fe7d48e9c15e admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:41.022689 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aecf2e8-58e2-43ff-bbbc-fe7d48e9c15e admin admin] PATH_INFO: `/v3/users` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:41.023310 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aecf2e8-58e2-43ff-bbbc-fe7d48e9c15e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:41.025201 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aecf2e8-58e2-43ff-bbbc-fe7d48e9c15e admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:41.115112 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 922/1852] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:19:40 2022] POST /identity/v3/users => generated 283 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:19:41.118922 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ce18e2-5da3-4a79-9b72-f69603ecf7d5 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:41.129938 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ce18e2-5da3-4a79-9b72-f69603ecf7d5 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:41.134559 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b907d8fd-5266-465e-adda-68c8db8bde7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:41.135164 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b907d8fd-5266-465e-adda-68c8db8bde7f admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:41.135442 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b907d8fd-5266-465e-adda-68c8db8bde7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:41.135712 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b907d8fd-5266-465e-adda-68c8db8bde7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:41.136358 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b907d8fd-5266-465e-adda-68c8db8bde7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:41.138059 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b907d8fd-5266-465e-adda-68c8db8bde7f admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:41.151063 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 931/1853] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:19:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:19:41.152219 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aecf2e8-58e2-43ff-bbbc-fe7d48e9c15e admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:41.162173 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aecf2e8-58e2-43ff-bbbc-fe7d48e9c15e admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:41.167141 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32462b4a-0d3f-49fe-9650-ba6399710c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:41.167750 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32462b4a-0d3f-49fe-9650-ba6399710c17 admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:41.168036 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32462b4a-0d3f-49fe-9650-ba6399710c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:41.168311 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32462b4a-0d3f-49fe-9650-ba6399710c17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:41.168993 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32462b4a-0d3f-49fe-9650-ba6399710c17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:41.170640 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32462b4a-0d3f-49fe-9650-ba6399710c17 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:41.184490 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 923/1854] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:19:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:19:41.185627 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b907d8fd-5266-465e-adda-68c8db8bde7f admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:41.196330 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b907d8fd-5266-465e-adda-68c8db8bde7f admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:41.201131 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7512877-e5f7-4321-93b9-065798a6537b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:41.201783 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7512877-e5f7-4321-93b9-065798a6537b admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:41.202066 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7512877-e5f7-4321-93b9-065798a6537b admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:41.202373 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7512877-e5f7-4321-93b9-065798a6537b admin admin] PATH_INFO: `/v3/projects/de1fb19b5e3e4e03acb2f6be518a8a82/users/a826cf4a5ac14d0d9714d37c2b58444f/roles/3255f77192524a16bc405173106871fc` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:41.216681 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7512877-e5f7-4321-93b9-065798a6537b admin admin] RBAC: Authorizing `identity:create_grant(project_id=de1fb19b5e3e4e03acb2f6be518a8a82, user_id=a826cf4a5ac14d0d9714d37c2b58444f, role_id=3255f77192524a16bc405173106871fc)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:41.219374 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7512877-e5f7-4321-93b9-065798a6537b admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:41.287151 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 932/1855] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:19:41 2022] PUT /identity/v3/projects/de1fb19b5e3e4e03acb2f6be518a8a82/users/a826cf4a5ac14d0d9714d37c2b58444f/roles/3255f77192524a16bc405173106871fc => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:19:41.300695 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fd23fd-a6ae-48e3-8234-be4b9a5eba02 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:41.301207 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fd23fd-a6ae-48e3-8234-be4b9a5eba02 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:41.302414 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fd23fd-a6ae-48e3-8234-be4b9a5eba02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:41.339932 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-c4fd23fd-a6ae-48e3-8234-be4b9a5eba02 None None] MFA Rules not processed for user `a826cf4a5ac14d0d9714d37c2b58444f`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:19:41.341350 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-c4fd23fd-a6ae-48e3-8234-be4b9a5eba02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:41.416802 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 924/1856] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:19:41 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:19:41.421327 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7512877-e5f7-4321-93b9-065798a6537b admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:41.483582 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7512877-e5f7-4321-93b9-065798a6537b admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:41.489594 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b9aa3b-369a-470b-b44b-c43b6e09a954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:41.490340 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b9aa3b-369a-470b-b44b-c43b6e09a954 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:41.490631 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b9aa3b-369a-470b-b44b-c43b6e09a954 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:41.490906 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b9aa3b-369a-470b-b44b-c43b6e09a954 admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:41.491548 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b9aa3b-369a-470b-b44b-c43b6e09a954 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:41.493454 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b9aa3b-369a-470b-b44b-c43b6e09a954 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:41.622410 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 933/1857] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:19:41 2022] POST /identity/v3/projects => generated 367 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:19:41.623549 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fd23fd-a6ae-48e3-8234-be4b9a5eba02 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:41.635978 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fd23fd-a6ae-48e3-8234-be4b9a5eba02 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:41.693968 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ca702f-6792-4a2e-b264-6ce91dd36192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:41.694693 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ca702f-6792-4a2e-b264-6ce91dd36192 admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:41.694883 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ca702f-6792-4a2e-b264-6ce91dd36192 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:41.695159 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ca702f-6792-4a2e-b264-6ce91dd36192 admin admin] PATH_INFO: `/v3/users` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:41.695788 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ca702f-6792-4a2e-b264-6ce91dd36192 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:41.697701 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ca702f-6792-4a2e-b264-6ce91dd36192 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:41.776956 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 925/1858] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:19:41 2022] POST /identity/v3/users => generated 290 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:19:41.781018 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b9aa3b-369a-470b-b44b-c43b6e09a954 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:41.791916 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b9aa3b-369a-470b-b44b-c43b6e09a954 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:41.796479 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83362246-6080-4885-b964-aa7ef79ad80d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:41.797071 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83362246-6080-4885-b964-aa7ef79ad80d admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:41.797347 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83362246-6080-4885-b964-aa7ef79ad80d admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:41.797646 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83362246-6080-4885-b964-aa7ef79ad80d admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:41.798300 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83362246-6080-4885-b964-aa7ef79ad80d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:41.800033 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83362246-6080-4885-b964-aa7ef79ad80d admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:41.801336 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:19:41.810945 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 934/1859] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:19:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:19:41.815092 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ca702f-6792-4a2e-b264-6ce91dd36192 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:41.826389 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ca702f-6792-4a2e-b264-6ce91dd36192 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:41.831591 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ad4c49-72a2-497b-a3f1-7dc7b5e10434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:41.832250 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ad4c49-72a2-497b-a3f1-7dc7b5e10434 admin admin] REQUEST_METHOD: `PUT` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:41.832527 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ad4c49-72a2-497b-a3f1-7dc7b5e10434 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:41.832800 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ad4c49-72a2-497b-a3f1-7dc7b5e10434 admin admin] PATH_INFO: `/v3/projects/3530c02d0d5a4e4ea6cdf9ec23fa12e6/users/03619407217c40c898ff3b2cdddc6416/roles/62f19ea272b447ccaab1576cd9bf4fd9` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:41.848073 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ad4c49-72a2-497b-a3f1-7dc7b5e10434 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3530c02d0d5a4e4ea6cdf9ec23fa12e6, user_id=03619407217c40c898ff3b2cdddc6416, role_id=62f19ea272b447ccaab1576cd9bf4fd9)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:41.850935 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ad4c49-72a2-497b-a3f1-7dc7b5e10434 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:41.899134 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:19:41.903771 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 926/1860] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:19:41 2022] PUT /identity/v3/projects/3530c02d0d5a4e4ea6cdf9ec23fa12e6/users/03619407217c40c898ff3b2cdddc6416/roles/62f19ea272b447ccaab1576cd9bf4fd9 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:19:41.908422 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83362246-6080-4885-b964-aa7ef79ad80d admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:41.918244 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83362246-6080-4885-b964-aa7ef79ad80d admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:41.973309 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162fc846-bc98-490f-9605-116e43134607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:41.973915 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162fc846-bc98-490f-9605-116e43134607 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:41.974196 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162fc846-bc98-490f-9605-116e43134607 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:41.974547 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162fc846-bc98-490f-9605-116e43134607 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:41.975220 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162fc846-bc98-490f-9605-116e43134607 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:41.976788 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162fc846-bc98-490f-9605-116e43134607 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:41.986330 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 935/1861] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:19:41 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:19:41.991360 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ad4c49-72a2-497b-a3f1-7dc7b5e10434 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:42.000732 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ad4c49-72a2-497b-a3f1-7dc7b5e10434 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:42.005571 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c5e0e9-e2e3-4f7c-ad10-1a0fc20dafb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:42.006231 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c5e0e9-e2e3-4f7c-ad10-1a0fc20dafb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:42.006563 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c5e0e9-e2e3-4f7c-ad10-1a0fc20dafb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:42.006843 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c5e0e9-e2e3-4f7c-ad10-1a0fc20dafb5 admin admin] PATH_INFO: `/v3/projects/3530c02d0d5a4e4ea6cdf9ec23fa12e6/users/03619407217c40c898ff3b2cdddc6416/roles/62f19ea272b447ccaab1576cd9bf4fd9` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:42.009264 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c5e0e9-e2e3-4f7c-ad10-1a0fc20dafb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3530c02d0d5a4e4ea6cdf9ec23fa12e6, user_id=03619407217c40c898ff3b2cdddc6416, role_id=62f19ea272b447ccaab1576cd9bf4fd9)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:42.011638 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c5e0e9-e2e3-4f7c-ad10-1a0fc20dafb5 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:42.022889 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 927/1862] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:19:41 2022] PUT /identity/v3/projects/3530c02d0d5a4e4ea6cdf9ec23fa12e6/users/03619407217c40c898ff3b2cdddc6416/roles/62f19ea272b447ccaab1576cd9bf4fd9 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:19:42.035069 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064e061a-c871-4b4c-83ca-f86db8b7d4e7 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:42.035359 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064e061a-c871-4b4c-83ca-f86db8b7d4e7 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:42.035635 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064e061a-c871-4b4c-83ca-f86db8b7d4e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:42.063994 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-064e061a-c871-4b4c-83ca-f86db8b7d4e7 None None] MFA Rules not processed for user `03619407217c40c898ff3b2cdddc6416`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:19:42.065281 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-064e061a-c871-4b4c-83ca-f86db8b7d4e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:42.132122 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 936/1863] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:19:42 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:19:42.144137 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c5e0e9-e2e3-4f7c-ad10-1a0fc20dafb5 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:42.145811 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-d7c5e0e9-e2e3-4f7c-ad10-1a0fc20dafb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:42.202788 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c5e0e9-e2e3-4f7c-ad10-1a0fc20dafb5 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:42.207025 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95651ef-0fd9-4c13-9097-3f92f10b23a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60193ba9e1c14df3bc9eebcc6f28f2d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:42.207631 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95651ef-0fd9-4c13-9097-3f92f10b23a8 service cinder] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:42.207911 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95651ef-0fd9-4c13-9097-3f92f10b23a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:42.208183 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95651ef-0fd9-4c13-9097-3f92f10b23a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:42.217171 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95651ef-0fd9-4c13-9097-3f92f10b23a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:42.218618 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95651ef-0fd9-4c13-9097-3f92f10b23a8 service cinder] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:42.227985 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 928/1864] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:19:42 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:19:42.434384 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064e061a-c871-4b4c-83ca-f86db8b7d4e7 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:42.444183 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064e061a-c871-4b4c-83ca-f86db8b7d4e7 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:42.447686 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3281247f-37b7-45fd-bc1b-f272b3f6c07a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60193ba9e1c14df3bc9eebcc6f28f2d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:42.448282 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3281247f-37b7-45fd-bc1b-f272b3f6c07a service cinder] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:42.448563 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3281247f-37b7-45fd-bc1b-f272b3f6c07a service cinder] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:42.449104 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3281247f-37b7-45fd-bc1b-f272b3f6c07a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:42.503805 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3281247f-37b7-45fd-bc1b-f272b3f6c07a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:42.505228 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3281247f-37b7-45fd-bc1b-f272b3f6c07a service cinder] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:42.513884 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 937/1865] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:19:42 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:19:42.803154 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95651ef-0fd9-4c13-9097-3f92f10b23a8 service cinder] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:42.807165 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-b95651ef-0fd9-4c13-9097-3f92f10b23a8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:42.894835 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95651ef-0fd9-4c13-9097-3f92f10b23a8 service cinder] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:42.903531 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1472f20-6bd0-4d18-8a3a-a6c67b31406b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:42.904432 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1472f20-6bd0-4d18-8a3a-a6c67b31406b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:42.905056 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1472f20-6bd0-4d18-8a3a-a6c67b31406b admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:42.905458 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1472f20-6bd0-4d18-8a3a-a6c67b31406b admin admin] PATH_INFO: `/v3/users/1086ed247eda4b0fbd8e5d36ec88f2d0` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:42.908884 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1472f20-6bd0-4d18-8a3a-a6c67b31406b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1086ed247eda4b0fbd8e5d36ec88f2d0)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:42.912080 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1472f20-6bd0-4d18-8a3a-a6c67b31406b admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:43.094959 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-b1472f20-6bd0-4d18-8a3a-a6c67b31406b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1086ed247eda4b0fbd8e5d36ec88f2d0'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:43.154805 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-b1472f20-6bd0-4d18-8a3a-a6c67b31406b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1086ed247eda4b0fbd8e5d36ec88f2d0'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:43.155443 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-b1472f20-6bd0-4d18-8a3a-a6c67b31406b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1086ed247eda4b0fbd8e5d36ec88f2d0'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:43.156242 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-b1472f20-6bd0-4d18-8a3a-a6c67b31406b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1086ed247eda4b0fbd8e5d36ec88f2d0'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:43.169579 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-b1472f20-6bd0-4d18-8a3a-a6c67b31406b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1086ed247eda4b0fbd8e5d36ec88f2d0'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:43.185477 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 929/1866] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:19:42 2022] DELETE /identity/v3/users/1086ed247eda4b0fbd8e5d36ec88f2d0 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:19:43.187402 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f37c45-7323-4112-859b-8c1950c6e0a6 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:43.187567 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f37c45-7323-4112-859b-8c1950c6e0a6 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:43.187843 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f37c45-7323-4112-859b-8c1950c6e0a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:43.207842 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-84f37c45-7323-4112-859b-8c1950c6e0a6 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:19:43.209055 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-84f37c45-7323-4112-859b-8c1950c6e0a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:43.218928 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:19:43.268173 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 938/1867] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:19:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:19:43.277782 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1472f20-6bd0-4d18-8a3a-a6c67b31406b admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:43.279244 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-b1472f20-6bd0-4d18-8a3a-a6c67b31406b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:43.290995 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:19:43.338153 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1472f20-6bd0-4d18-8a3a-a6c67b31406b admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:43.341817 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef8ad3d-032c-4837-b3e8-bfb7576cbae2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:43.342829 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef8ad3d-032c-4837-b3e8-bfb7576cbae2 service neutron] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:43.343309 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef8ad3d-032c-4837-b3e8-bfb7576cbae2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:43.343950 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef8ad3d-032c-4837-b3e8-bfb7576cbae2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:43.353870 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef8ad3d-032c-4837-b3e8-bfb7576cbae2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:43.354885 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef8ad3d-032c-4837-b3e8-bfb7576cbae2 service neutron] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:43.364557 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 930/1868] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:19:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:19:43.750077 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f37c45-7323-4112-859b-8c1950c6e0a6 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:43.752369 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-84f37c45-7323-4112-859b-8c1950c6e0a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:43.767264 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f37c45-7323-4112-859b-8c1950c6e0a6 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:43.772279 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8153681d-9e38-4aaa-89d3-6bf5cdeb7840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:43.772939 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8153681d-9e38-4aaa-89d3-6bf5cdeb7840 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:43.773252 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8153681d-9e38-4aaa-89d3-6bf5cdeb7840 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:43.773541 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8153681d-9e38-4aaa-89d3-6bf5cdeb7840 admin admin] PATH_INFO: `/v3/projects/e87afdbba324458198f8e53262bbac65` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:43.774668 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8153681d-9e38-4aaa-89d3-6bf5cdeb7840 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e87afdbba324458198f8e53262bbac65)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:43.776679 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8153681d-9e38-4aaa-89d3-6bf5cdeb7840 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:43.888087 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-8153681d-9e38-4aaa-89d3-6bf5cdeb7840 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e87afdbba324458198f8e53262bbac65', 'request_id': 'req-8153681d-9e38-4aaa-89d3-6bf5cdeb7840'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:43.977712 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-8153681d-9e38-4aaa-89d3-6bf5cdeb7840 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e87afdbba324458198f8e53262bbac65', 'request_id': 'req-8153681d-9e38-4aaa-89d3-6bf5cdeb7840'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:43.986533 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-8153681d-9e38-4aaa-89d3-6bf5cdeb7840 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e87afdbba324458198f8e53262bbac65', 'request_id': 'req-8153681d-9e38-4aaa-89d3-6bf5cdeb7840'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:44.001846 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-8153681d-9e38-4aaa-89d3-6bf5cdeb7840 admin admin] The token cache is being invalidate because project e87afdbba324458198f8e53262bbac65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:19:44.002590 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-8153681d-9e38-4aaa-89d3-6bf5cdeb7840 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:44.003286 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-8153681d-9e38-4aaa-89d3-6bf5cdeb7840 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:44.005192 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 939/1869] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:19:43 2022] DELETE /identity/v3/projects/e87afdbba324458198f8e53262bbac65 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:19:44.008904 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef8ad3d-032c-4837-b3e8-bfb7576cbae2 service neutron] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:44.024419 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef8ad3d-032c-4837-b3e8-bfb7576cbae2 service neutron] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:44.099963 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63556746-573f-406f-8bf5-88464b90fb75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:44.100619 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63556746-573f-406f-8bf5-88464b90fb75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:44.100898 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63556746-573f-406f-8bf5-88464b90fb75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:44.101177 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63556746-573f-406f-8bf5-88464b90fb75 admin admin] PATH_INFO: `/v3/users/6f9c19326ccf4ecf91ee6730dbfdf7b7` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:44.102382 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63556746-573f-406f-8bf5-88464b90fb75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f9c19326ccf4ecf91ee6730dbfdf7b7)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:44.104320 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63556746-573f-406f-8bf5-88464b90fb75 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:44.281900 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-63556746-573f-406f-8bf5-88464b90fb75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f9c19326ccf4ecf91ee6730dbfdf7b7'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:44.331457 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-63556746-573f-406f-8bf5-88464b90fb75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f9c19326ccf4ecf91ee6730dbfdf7b7'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:44.332016 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-63556746-573f-406f-8bf5-88464b90fb75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f9c19326ccf4ecf91ee6730dbfdf7b7'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:44.332624 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-63556746-573f-406f-8bf5-88464b90fb75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f9c19326ccf4ecf91ee6730dbfdf7b7'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:44.344920 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-63556746-573f-406f-8bf5-88464b90fb75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f9c19326ccf4ecf91ee6730dbfdf7b7'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:44.359916 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 931/1870] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:19:44 2022] DELETE /identity/v3/users/6f9c19326ccf4ecf91ee6730dbfdf7b7 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:19:45.117440 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8153681d-9e38-4aaa-89d3-6bf5cdeb7840 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:45.119066 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-8153681d-9e38-4aaa-89d3-6bf5cdeb7840 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:45.144624 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8153681d-9e38-4aaa-89d3-6bf5cdeb7840 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:45.228878 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8b4b69-6676-4fd8-a192-a17e6f3f9ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:45.229663 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8b4b69-6676-4fd8-a192-a17e6f3f9ef1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:45.230260 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8b4b69-6676-4fd8-a192-a17e6f3f9ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:45.230457 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8b4b69-6676-4fd8-a192-a17e6f3f9ef1 admin admin] PATH_INFO: `/v3/projects/02586873b0a549f393d8653d449e591f` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:45.232165 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8b4b69-6676-4fd8-a192-a17e6f3f9ef1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02586873b0a549f393d8653d449e591f)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:45.233934 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8b4b69-6676-4fd8-a192-a17e6f3f9ef1 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:45.369700 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:19:45.377712 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-3a8b4b69-6676-4fd8-a192-a17e6f3f9ef1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02586873b0a549f393d8653d449e591f', 'request_id': 'req-3a8b4b69-6676-4fd8-a192-a17e6f3f9ef1'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:45.486509 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-3a8b4b69-6676-4fd8-a192-a17e6f3f9ef1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02586873b0a549f393d8653d449e591f', 'request_id': 'req-3a8b4b69-6676-4fd8-a192-a17e6f3f9ef1'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:45.495336 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-3a8b4b69-6676-4fd8-a192-a17e6f3f9ef1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02586873b0a549f393d8653d449e591f', 'request_id': 'req-3a8b4b69-6676-4fd8-a192-a17e6f3f9ef1'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:45.509345 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-3a8b4b69-6676-4fd8-a192-a17e6f3f9ef1 admin admin] The token cache is being invalidate because project 02586873b0a549f393d8653d449e591f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:19:45.509640 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-3a8b4b69-6676-4fd8-a192-a17e6f3f9ef1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:45.510324 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-3a8b4b69-6676-4fd8-a192-a17e6f3f9ef1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:45.511944 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 940/1871] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:19:45 2022] DELETE /identity/v3/projects/02586873b0a549f393d8653d449e591f => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:19:45.517019 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5350e6-1531-4c44-8944-6a3d859406ad None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:45.517511 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5350e6-1531-4c44-8944-6a3d859406ad None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:45.517601 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5350e6-1531-4c44-8944-6a3d859406ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:45.537406 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-5a5350e6-1531-4c44-8944-6a3d859406ad None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:19:45.538686 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-5a5350e6-1531-4c44-8944-6a3d859406ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:45.583787 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:19:45.613391 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 932/1872] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:19:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:19:45.621540 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813dbd44-3f15-43ec-b56b-58305e0c5de4 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:45.621763 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813dbd44-3f15-43ec-b56b-58305e0c5de4 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:45.622041 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813dbd44-3f15-43ec-b56b-58305e0c5de4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:45.643039 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-813dbd44-3f15-43ec-b56b-58305e0c5de4 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:19:45.644264 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-813dbd44-3f15-43ec-b56b-58305e0c5de4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:45.652269 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 941/1873] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:19:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:19:45.656726 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5350e6-1531-4c44-8944-6a3d859406ad None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:45.668087 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5350e6-1531-4c44-8944-6a3d859406ad None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:45.674797 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9af093-f5f0-40d8-bdf1-246e24988fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:45.675540 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9af093-f5f0-40d8-bdf1-246e24988fd7 admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:45.675920 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9af093-f5f0-40d8-bdf1-246e24988fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:45.676303 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9af093-f5f0-40d8-bdf1-246e24988fd7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:45.677020 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9af093-f5f0-40d8-bdf1-246e24988fd7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:19:45.677389 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9af093-f5f0-40d8-bdf1-246e24988fd7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:45.680507 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9af093-f5f0-40d8-bdf1-246e24988fd7 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:45.696770 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 933/1874] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Aug 25 03:19:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:19:45.701035 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813dbd44-3f15-43ec-b56b-58305e0c5de4 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:45.710243 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813dbd44-3f15-43ec-b56b-58305e0c5de4 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:45.715030 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9075511-3a86-4c56-9549-5b9bd8b3d38f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:45.715650 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9075511-3a86-4c56-9549-5b9bd8b3d38f admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:45.715932 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9075511-3a86-4c56-9549-5b9bd8b3d38f admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:45.716207 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9075511-3a86-4c56-9549-5b9bd8b3d38f admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:45.716844 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9075511-3a86-4c56-9549-5b9bd8b3d38f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:45.718784 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9075511-3a86-4c56-9549-5b9bd8b3d38f admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:45.859855 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9af093-f5f0-40d8-bdf1-246e24988fd7 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:45.861679 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 942/1875] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:19:45 2022] POST /identity/v3/projects => generated 381 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:19:45.881119 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9af093-f5f0-40d8-bdf1-246e24988fd7 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:45.991396 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641e4321-27a8-4409-ba12-deaa0db54d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:45.992192 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641e4321-27a8-4409-ba12-deaa0db54d73 admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:45.993105 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641e4321-27a8-4409-ba12-deaa0db54d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:45.993978 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641e4321-27a8-4409-ba12-deaa0db54d73 admin admin] PATH_INFO: `/v3/users` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:45.994725 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641e4321-27a8-4409-ba12-deaa0db54d73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:45.997300 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641e4321-27a8-4409-ba12-deaa0db54d73 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:46.103137 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 934/1876] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:19:45 2022] POST /identity/v3/users => generated 291 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:19:46.105217 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9075511-3a86-4c56-9549-5b9bd8b3d38f admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:46.117961 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9075511-3a86-4c56-9549-5b9bd8b3d38f admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:46.122896 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb00e77-2bf2-496f-8d33-3979a12892c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:46.123559 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb00e77-2bf2-496f-8d33-3979a12892c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:46.123838 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb00e77-2bf2-496f-8d33-3979a12892c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:46.124111 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb00e77-2bf2-496f-8d33-3979a12892c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:46.124908 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb00e77-2bf2-496f-8d33-3979a12892c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:46.126626 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb00e77-2bf2-496f-8d33-3979a12892c9 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:46.137520 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 943/1877] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:19:46 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:19:46.141727 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641e4321-27a8-4409-ba12-deaa0db54d73 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:46.152150 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641e4321-27a8-4409-ba12-deaa0db54d73 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:46.157676 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b70b54-b69a-4a92-8982-f0c1207841d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:46.158307 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b70b54-b69a-4a92-8982-f0c1207841d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:46.158590 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b70b54-b69a-4a92-8982-f0c1207841d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:46.158866 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b70b54-b69a-4a92-8982-f0c1207841d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:46.159520 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b70b54-b69a-4a92-8982-f0c1207841d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:46.161192 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b70b54-b69a-4a92-8982-f0c1207841d6 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:46.172018 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 935/1878] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:19:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:19:46.176961 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb00e77-2bf2-496f-8d33-3979a12892c9 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:46.186817 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb00e77-2bf2-496f-8d33-3979a12892c9 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:46.191657 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6577eb6f-bd12-40b8-bc22-7a866f5fca54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:46.192557 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6577eb6f-bd12-40b8-bc22-7a866f5fca54 admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:46.192901 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6577eb6f-bd12-40b8-bc22-7a866f5fca54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:46.193181 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6577eb6f-bd12-40b8-bc22-7a866f5fca54 admin admin] PATH_INFO: `/v3/projects/94a25b21191341f39080554fdfe55e65/users/e501606859504c89a896d6b19afbb21a/roles/3255f77192524a16bc405173106871fc` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:46.209942 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6577eb6f-bd12-40b8-bc22-7a866f5fca54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94a25b21191341f39080554fdfe55e65, user_id=e501606859504c89a896d6b19afbb21a, role_id=3255f77192524a16bc405173106871fc)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:46.212997 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6577eb6f-bd12-40b8-bc22-7a866f5fca54 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:46.277246 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 944/1879] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:19:46 2022] PUT /identity/v3/projects/94a25b21191341f39080554fdfe55e65/users/e501606859504c89a896d6b19afbb21a/roles/3255f77192524a16bc405173106871fc => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:19:46.289785 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7e216d-f4dc-47ab-b827-66281041c0cb None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:46.290241 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7e216d-f4dc-47ab-b827-66281041c0cb None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:46.290531 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7e216d-f4dc-47ab-b827-66281041c0cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:46.319327 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-6e7e216d-f4dc-47ab-b827-66281041c0cb None None] MFA Rules not processed for user `e501606859504c89a896d6b19afbb21a`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:19:46.322042 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-6e7e216d-f4dc-47ab-b827-66281041c0cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:46.394482 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 936/1880] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:19:46 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:19:46.399331 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6577eb6f-bd12-40b8-bc22-7a866f5fca54 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:46.464996 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6577eb6f-bd12-40b8-bc22-7a866f5fca54 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:46.470917 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee99309f-e07e-45c6-be97-c2726eefd739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:46.471762 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee99309f-e07e-45c6-be97-c2726eefd739 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:46.472051 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee99309f-e07e-45c6-be97-c2726eefd739 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:46.472325 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee99309f-e07e-45c6-be97-c2726eefd739 admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:46.473075 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee99309f-e07e-45c6-be97-c2726eefd739 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:46.475015 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee99309f-e07e-45c6-be97-c2726eefd739 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:46.605907 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 945/1881] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:19:46 2022] POST /identity/v3/projects => generated 379 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:19:46.610675 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7e216d-f4dc-47ab-b827-66281041c0cb None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:46.623998 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7e216d-f4dc-47ab-b827-66281041c0cb None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:46.655887 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:19:46.706487 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44a8625-17fc-4685-b6e0-ad68f276e16c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:46.709166 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44a8625-17fc-4685-b6e0-ad68f276e16c admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:46.709166 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44a8625-17fc-4685-b6e0-ad68f276e16c admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:46.709166 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44a8625-17fc-4685-b6e0-ad68f276e16c admin admin] PATH_INFO: `/v3/users` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:46.709166 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44a8625-17fc-4685-b6e0-ad68f276e16c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:46.711851 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44a8625-17fc-4685-b6e0-ad68f276e16c admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:46.836730 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 937/1882] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:19:46 2022] POST /identity/v3/users => generated 296 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:19:46.840866 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee99309f-e07e-45c6-be97-c2726eefd739 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:46.852798 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee99309f-e07e-45c6-be97-c2726eefd739 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:46.862575 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4088cf57-5778-4619-9854-89273a6bf01e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:46.863261 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4088cf57-5778-4619-9854-89273a6bf01e admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:46.863554 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4088cf57-5778-4619-9854-89273a6bf01e admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:46.872156 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4088cf57-5778-4619-9854-89273a6bf01e admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:46.872156 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4088cf57-5778-4619-9854-89273a6bf01e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:46.872156 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4088cf57-5778-4619-9854-89273a6bf01e admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:46.877497 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 946/1883] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:19:46 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:19:46.881784 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44a8625-17fc-4685-b6e0-ad68f276e16c admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:46.892542 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44a8625-17fc-4685-b6e0-ad68f276e16c admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:46.897315 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dac6c0-2811-4183-a005-6646ec70dd36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:46.898004 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dac6c0-2811-4183-a005-6646ec70dd36 admin admin] REQUEST_METHOD: `PUT` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:46.898308 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dac6c0-2811-4183-a005-6646ec70dd36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:46.898591 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dac6c0-2811-4183-a005-6646ec70dd36 admin admin] PATH_INFO: `/v3/projects/0c0fc58f428c44e6a8c8556917a38403/users/bb13f202fabd4ffe838fceecb6613b70/roles/62f19ea272b447ccaab1576cd9bf4fd9` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:46.913070 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dac6c0-2811-4183-a005-6646ec70dd36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c0fc58f428c44e6a8c8556917a38403, user_id=bb13f202fabd4ffe838fceecb6613b70, role_id=62f19ea272b447ccaab1576cd9bf4fd9)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:46.915775 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dac6c0-2811-4183-a005-6646ec70dd36 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:47.025093 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 938/1884] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:19:46 2022] PUT /identity/v3/projects/0c0fc58f428c44e6a8c8556917a38403/users/bb13f202fabd4ffe838fceecb6613b70/roles/62f19ea272b447ccaab1576cd9bf4fd9 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:19:47.029020 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4088cf57-5778-4619-9854-89273a6bf01e admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:47.039328 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4088cf57-5778-4619-9854-89273a6bf01e admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:47.080508 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:19:47.099570 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15379ad-14be-47d7-ba61-28f8dd43f2d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:47.100208 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15379ad-14be-47d7-ba61-28f8dd43f2d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:47.100488 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15379ad-14be-47d7-ba61-28f8dd43f2d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:47.100762 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15379ad-14be-47d7-ba61-28f8dd43f2d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:47.101409 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15379ad-14be-47d7-ba61-28f8dd43f2d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:47.103031 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15379ad-14be-47d7-ba61-28f8dd43f2d0 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:47.113438 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 947/1885] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:19:47 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:19:47.117606 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dac6c0-2811-4183-a005-6646ec70dd36 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:47.128513 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dac6c0-2811-4183-a005-6646ec70dd36 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:47.133308 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60664d0-26f4-456d-b80f-12932f3ac4aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:47.140687 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60664d0-26f4-456d-b80f-12932f3ac4aa admin admin] REQUEST_METHOD: `PUT` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:47.140687 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60664d0-26f4-456d-b80f-12932f3ac4aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:47.140687 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60664d0-26f4-456d-b80f-12932f3ac4aa admin admin] PATH_INFO: `/v3/projects/0c0fc58f428c44e6a8c8556917a38403/users/bb13f202fabd4ffe838fceecb6613b70/roles/62f19ea272b447ccaab1576cd9bf4fd9` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:47.140687 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60664d0-26f4-456d-b80f-12932f3ac4aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c0fc58f428c44e6a8c8556917a38403, user_id=bb13f202fabd4ffe838fceecb6613b70, role_id=62f19ea272b447ccaab1576cd9bf4fd9)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:47.140687 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60664d0-26f4-456d-b80f-12932f3ac4aa admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:47.152800 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 939/1886] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:19:47 2022] PUT /identity/v3/projects/0c0fc58f428c44e6a8c8556917a38403/users/bb13f202fabd4ffe838fceecb6613b70/roles/62f19ea272b447ccaab1576cd9bf4fd9 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:19:47.161598 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d3a7e6-f699-4349-9256-2c6d07996128 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:47.162193 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d3a7e6-f699-4349-9256-2c6d07996128 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:47.162193 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d3a7e6-f699-4349-9256-2c6d07996128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:47.188044 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60664d0-26f4-456d-b80f-12932f3ac4aa admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:47.190437 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-e60664d0-26f4-456d-b80f-12932f3ac4aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:47.196932 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-98d3a7e6-f699-4349-9256-2c6d07996128 None None] MFA Rules not processed for user `bb13f202fabd4ffe838fceecb6613b70`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:19:47.198229 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-98d3a7e6-f699-4349-9256-2c6d07996128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:47.263372 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60664d0-26f4-456d-b80f-12932f3ac4aa admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:47.268496 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b6483a-defe-4a0b-8057-92370d11a9ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:47.269129 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b6483a-defe-4a0b-8057-92370d11a9ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:47.269402 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b6483a-defe-4a0b-8057-92370d11a9ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:47.269670 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b6483a-defe-4a0b-8057-92370d11a9ae admin admin] PATH_INFO: `/v3/users/752712f3bbab451fa13b419bcf4f3f71` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:47.270820 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b6483a-defe-4a0b-8057-92370d11a9ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=752712f3bbab451fa13b419bcf4f3f71)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:47.272734 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b6483a-defe-4a0b-8057-92370d11a9ae admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:47.275541 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 948/1887] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:19:47 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:19:47.287247 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d3a7e6-f699-4349-9256-2c6d07996128 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:47.288774 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-98d3a7e6-f699-4349-9256-2c6d07996128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:47.350151 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d3a7e6-f699-4349-9256-2c6d07996128 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:47.354047 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aaf6d0-06b7-47fe-a85e-165dc5a5d4ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60193ba9e1c14df3bc9eebcc6f28f2d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:47.354949 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aaf6d0-06b7-47fe-a85e-165dc5a5d4ca service cinder] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:47.355572 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aaf6d0-06b7-47fe-a85e-165dc5a5d4ca service cinder] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:47.355572 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aaf6d0-06b7-47fe-a85e-165dc5a5d4ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:47.366089 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aaf6d0-06b7-47fe-a85e-165dc5a5d4ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:47.367638 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aaf6d0-06b7-47fe-a85e-165dc5a5d4ca service cinder] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:47.377251 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 949/1888] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:19:47 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 25 03:19:47.443419 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-b0b6483a-defe-4a0b-8057-92370d11a9ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '752712f3bbab451fa13b419bcf4f3f71'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:47.508914 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-b0b6483a-defe-4a0b-8057-92370d11a9ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '752712f3bbab451fa13b419bcf4f3f71'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:47.509611 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-b0b6483a-defe-4a0b-8057-92370d11a9ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '752712f3bbab451fa13b419bcf4f3f71'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:47.510267 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-b0b6483a-defe-4a0b-8057-92370d11a9ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '752712f3bbab451fa13b419bcf4f3f71'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:47.523001 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-b0b6483a-defe-4a0b-8057-92370d11a9ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '752712f3bbab451fa13b419bcf4f3f71'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:47.530694 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:19:47.536172 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 940/1889] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:19:47 2022] DELETE /identity/v3/users/752712f3bbab451fa13b419bcf4f3f71 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:19:47.540707 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0d823f-c4d9-44c3-b7a2-7a2f5caee82f None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:47.540993 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0d823f-c4d9-44c3-b7a2-7a2f5caee82f None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:47.541304 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0d823f-c4d9-44c3-b7a2-7a2f5caee82f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:47.566925 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-1e0d823f-c4d9-44c3-b7a2-7a2f5caee82f None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:19:47.566925 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-1e0d823f-c4d9-44c3-b7a2-7a2f5caee82f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:47.574485 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b6483a-defe-4a0b-8057-92370d11a9ae admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:47.575949 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-b0b6483a-defe-4a0b-8057-92370d11a9ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:47.649439 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b6483a-defe-4a0b-8057-92370d11a9ae admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:47.649949 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 950/1890] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:19:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:19:47.650047 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca82281-3fb6-4e94-92d1-8eaed0baa56b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60193ba9e1c14df3bc9eebcc6f28f2d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:47.650047 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca82281-3fb6-4e94-92d1-8eaed0baa56b service cinder] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:47.650047 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca82281-3fb6-4e94-92d1-8eaed0baa56b service cinder] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:47.650047 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca82281-3fb6-4e94-92d1-8eaed0baa56b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:47.652003 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-3ca82281-3fb6-4e94-92d1-8eaed0baa56b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:47.658878 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0d823f-c4d9-44c3-b7a2-7a2f5caee82f None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:47.660587 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-1e0d823f-c4d9-44c3-b7a2-7a2f5caee82f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:47.667394 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:19:47.728998 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0d823f-c4d9-44c3-b7a2-7a2f5caee82f None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:47.732857 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca82281-3fb6-4e94-92d1-8eaed0baa56b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:47.734441 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca82281-3fb6-4e94-92d1-8eaed0baa56b service cinder] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:47.741026 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6c8ccf-be12-430c-a015-83113a6fd8d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:47.741026 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6c8ccf-be12-430c-a015-83113a6fd8d4 service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:47.741026 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6c8ccf-be12-430c-a015-83113a6fd8d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:47.741026 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6c8ccf-be12-430c-a015-83113a6fd8d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:47.748478 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6c8ccf-be12-430c-a015-83113a6fd8d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:47.750073 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6c8ccf-be12-430c-a015-83113a6fd8d4 service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:47.751984 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 941/1891] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:19:47 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 25 03:19:47.777288 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 951/1892] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:19:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 25 03:19:48.248601 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca82281-3fb6-4e94-92d1-8eaed0baa56b service cinder] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:48.257427 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-3ca82281-3fb6-4e94-92d1-8eaed0baa56b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:48.265031 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca82281-3fb6-4e94-92d1-8eaed0baa56b service cinder] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:48.271535 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35d6d0b-d5d6-4d44-ad17-4f1cb040d74a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:48.276115 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35d6d0b-d5d6-4d44-ad17-4f1cb040d74a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:48.276115 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35d6d0b-d5d6-4d44-ad17-4f1cb040d74a admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:48.276115 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35d6d0b-d5d6-4d44-ad17-4f1cb040d74a admin admin] PATH_INFO: `/v3/projects/17ec8c3525614c4bba01263430d6c32b` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:48.276359 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35d6d0b-d5d6-4d44-ad17-4f1cb040d74a admin admin] RBAC: Authorizing `identity:delete_project(project_id=17ec8c3525614c4bba01263430d6c32b)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:48.280369 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35d6d0b-d5d6-4d44-ad17-4f1cb040d74a admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:48.395264 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-f35d6d0b-d5d6-4d44-ad17-4f1cb040d74a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17ec8c3525614c4bba01263430d6c32b', 'request_id': 'req-f35d6d0b-d5d6-4d44-ad17-4f1cb040d74a'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:48.480403 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-f35d6d0b-d5d6-4d44-ad17-4f1cb040d74a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17ec8c3525614c4bba01263430d6c32b', 'request_id': 'req-f35d6d0b-d5d6-4d44-ad17-4f1cb040d74a'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:48.487956 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-f35d6d0b-d5d6-4d44-ad17-4f1cb040d74a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17ec8c3525614c4bba01263430d6c32b', 'request_id': 'req-f35d6d0b-d5d6-4d44-ad17-4f1cb040d74a'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:48.499274 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-f35d6d0b-d5d6-4d44-ad17-4f1cb040d74a admin admin] The token cache is being invalidate because project 17ec8c3525614c4bba01263430d6c32b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81995) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:19:48.499566 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-f35d6d0b-d5d6-4d44-ad17-4f1cb040d74a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:48.500356 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-f35d6d0b-d5d6-4d44-ad17-4f1cb040d74a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:48.502545 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 942/1893] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:19:48 2022] DELETE /identity/v3/projects/17ec8c3525614c4bba01263430d6c32b => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:19:48.506651 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6c8ccf-be12-430c-a015-83113a6fd8d4 service neutron] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:48.516272 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6c8ccf-be12-430c-a015-83113a6fd8d4 service neutron] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:48.574379 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fb74f0-4590-4396-bdeb-ce05670daaf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:48.575030 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fb74f0-4590-4396-bdeb-ce05670daaf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:48.575307 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fb74f0-4590-4396-bdeb-ce05670daaf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:48.575581 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fb74f0-4590-4396-bdeb-ce05670daaf7 admin admin] PATH_INFO: `/v3/users/57aa1499b90c48fda1c38c96f69a755c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:48.576704 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fb74f0-4590-4396-bdeb-ce05670daaf7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=57aa1499b90c48fda1c38c96f69a755c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:48.578723 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fb74f0-4590-4396-bdeb-ce05670daaf7 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:48.752248 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-55fb74f0-4590-4396-bdeb-ce05670daaf7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57aa1499b90c48fda1c38c96f69a755c'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:48.806009 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-55fb74f0-4590-4396-bdeb-ce05670daaf7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57aa1499b90c48fda1c38c96f69a755c'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:48.806658 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-55fb74f0-4590-4396-bdeb-ce05670daaf7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57aa1499b90c48fda1c38c96f69a755c'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:48.814684 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-55fb74f0-4590-4396-bdeb-ce05670daaf7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57aa1499b90c48fda1c38c96f69a755c'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:48.826685 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-55fb74f0-4590-4396-bdeb-ce05670daaf7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57aa1499b90c48fda1c38c96f69a755c'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:48.832558 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 952/1894] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:19:48 2022] DELETE /identity/v3/users/57aa1499b90c48fda1c38c96f69a755c => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:19:49.304059 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35d6d0b-d5d6-4d44-ad17-4f1cb040d74a admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:49.305990 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-f35d6d0b-d5d6-4d44-ad17-4f1cb040d74a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:49.319278 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35d6d0b-d5d6-4d44-ad17-4f1cb040d74a admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:49.355240 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:19:49.379034 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e0a95b-b611-4df4-9fe3-65287f04d54b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:49.379667 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e0a95b-b611-4df4-9fe3-65287f04d54b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:49.379950 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e0a95b-b611-4df4-9fe3-65287f04d54b admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:49.380307 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e0a95b-b611-4df4-9fe3-65287f04d54b admin admin] PATH_INFO: `/v3/projects/feaf957becd94a3ea8544c906b37ddfc` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:49.381427 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e0a95b-b611-4df4-9fe3-65287f04d54b admin admin] RBAC: Authorizing `identity:delete_project(project_id=feaf957becd94a3ea8544c906b37ddfc)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:49.383368 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e0a95b-b611-4df4-9fe3-65287f04d54b admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:49.503200 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-76e0a95b-b611-4df4-9fe3-65287f04d54b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'feaf957becd94a3ea8544c906b37ddfc', 'request_id': 'req-76e0a95b-b611-4df4-9fe3-65287f04d54b'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:49.565928 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-76e0a95b-b611-4df4-9fe3-65287f04d54b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'feaf957becd94a3ea8544c906b37ddfc', 'request_id': 'req-76e0a95b-b611-4df4-9fe3-65287f04d54b'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:49.571947 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-76e0a95b-b611-4df4-9fe3-65287f04d54b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'feaf957becd94a3ea8544c906b37ddfc', 'request_id': 'req-76e0a95b-b611-4df4-9fe3-65287f04d54b'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:49.584051 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-76e0a95b-b611-4df4-9fe3-65287f04d54b admin admin] The token cache is being invalidate because project feaf957becd94a3ea8544c906b37ddfc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81995) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:19:49.584344 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-76e0a95b-b611-4df4-9fe3-65287f04d54b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:49.584990 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-76e0a95b-b611-4df4-9fe3-65287f04d54b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:49.586533 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 943/1895] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:19:49 2022] DELETE /identity/v3/projects/feaf957becd94a3ea8544c906b37ddfc => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:19:49.590490 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fb74f0-4590-4396-bdeb-ce05670daaf7 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:49.591955 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-55fb74f0-4590-4396-bdeb-ce05670daaf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:49.604686 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fb74f0-4590-4396-bdeb-ce05670daaf7 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:49.629349 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:19:49.662654 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020a8182-573d-4bd2-8922-35dda4adc81b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:49.663307 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020a8182-573d-4bd2-8922-35dda4adc81b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:49.663591 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020a8182-573d-4bd2-8922-35dda4adc81b admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:49.663868 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020a8182-573d-4bd2-8922-35dda4adc81b admin admin] PATH_INFO: `/v3/users/030de938bb44423f9da013f081022a90` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:49.665009 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020a8182-573d-4bd2-8922-35dda4adc81b admin admin] RBAC: Authorizing `identity:delete_user(user_id=030de938bb44423f9da013f081022a90)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:49.666972 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020a8182-573d-4bd2-8922-35dda4adc81b admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:49.807595 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-020a8182-573d-4bd2-8922-35dda4adc81b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '030de938bb44423f9da013f081022a90'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:49.863574 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-020a8182-573d-4bd2-8922-35dda4adc81b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '030de938bb44423f9da013f081022a90'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:49.864191 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-020a8182-573d-4bd2-8922-35dda4adc81b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '030de938bb44423f9da013f081022a90'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:49.873308 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-020a8182-573d-4bd2-8922-35dda4adc81b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '030de938bb44423f9da013f081022a90'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:49.889933 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-020a8182-573d-4bd2-8922-35dda4adc81b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '030de938bb44423f9da013f081022a90'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:49.896786 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 953/1896] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:19:49 2022] DELETE /identity/v3/users/030de938bb44423f9da013f081022a90 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:19:50.380050 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e0a95b-b611-4df4-9fe3-65287f04d54b admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:50.382105 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-76e0a95b-b611-4df4-9fe3-65287f04d54b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:50.395305 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e0a95b-b611-4df4-9fe3-65287f04d54b admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:50.466253 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470bbd17-a9ee-4edf-bfd0-abd22747ac6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:50.467355 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470bbd17-a9ee-4edf-bfd0-abd22747ac6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:50.467765 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470bbd17-a9ee-4edf-bfd0-abd22747ac6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:50.468186 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470bbd17-a9ee-4edf-bfd0-abd22747ac6b admin admin] PATH_INFO: `/v3/projects/2e0db11694ce4317881f103ab8a659ad` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:50.469380 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470bbd17-a9ee-4edf-bfd0-abd22747ac6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e0db11694ce4317881f103ab8a659ad)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:50.472818 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470bbd17-a9ee-4edf-bfd0-abd22747ac6b admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:50.592053 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-470bbd17-a9ee-4edf-bfd0-abd22747ac6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e0db11694ce4317881f103ab8a659ad', 'request_id': 'req-470bbd17-a9ee-4edf-bfd0-abd22747ac6b'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:50.663282 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-470bbd17-a9ee-4edf-bfd0-abd22747ac6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e0db11694ce4317881f103ab8a659ad', 'request_id': 'req-470bbd17-a9ee-4edf-bfd0-abd22747ac6b'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:50.669325 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-470bbd17-a9ee-4edf-bfd0-abd22747ac6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e0db11694ce4317881f103ab8a659ad', 'request_id': 'req-470bbd17-a9ee-4edf-bfd0-abd22747ac6b'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:50.671233 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:19:50.680870 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-470bbd17-a9ee-4edf-bfd0-abd22747ac6b admin admin] The token cache is being invalidate because project 2e0db11694ce4317881f103ab8a659ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81995) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:19:50.681162 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-470bbd17-a9ee-4edf-bfd0-abd22747ac6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:50.681815 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-470bbd17-a9ee-4edf-bfd0-abd22747ac6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:50.683373 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 944/1897] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:19:50 2022] DELETE /identity/v3/projects/2e0db11694ce4317881f103ab8a659ad => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:19:50.690012 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0c229d-d13c-4596-ae96-e26a0bc33904 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:50.690485 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0c229d-d13c-4596-ae96-e26a0bc33904 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:50.690603 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0c229d-d13c-4596-ae96-e26a0bc33904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:50.710011 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-2a0c229d-d13c-4596-ae96-e26a0bc33904 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:19:50.711265 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-2a0c229d-d13c-4596-ae96-e26a0bc33904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:50.771383 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 954/1898] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:19:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:19:50.780256 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a255bb52-7093-47df-998c-c8badf5043ea None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:50.780694 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a255bb52-7093-47df-998c-c8badf5043ea None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:50.780853 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a255bb52-7093-47df-998c-c8badf5043ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:50.801879 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-a255bb52-7093-47df-998c-c8badf5043ea None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:19:50.803164 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-a255bb52-7093-47df-998c-c8badf5043ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:50.811442 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 945/1899] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:19:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:19:50.816462 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0c229d-d13c-4596-ae96-e26a0bc33904 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:50.827306 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0c229d-d13c-4596-ae96-e26a0bc33904 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:50.832094 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6cabbb-e7f2-49e0-80ea-21d5da614963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:50.832743 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6cabbb-e7f2-49e0-80ea-21d5da614963 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:50.833023 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6cabbb-e7f2-49e0-80ea-21d5da614963 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:50.833298 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6cabbb-e7f2-49e0-80ea-21d5da614963 admin admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:50.833963 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6cabbb-e7f2-49e0-80ea-21d5da614963 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:19:50.834267 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6cabbb-e7f2-49e0-80ea-21d5da614963 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:50.835940 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6cabbb-e7f2-49e0-80ea-21d5da614963 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:50.852970 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 955/1900] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Aug 25 03:19:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:19:50.854471 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a255bb52-7093-47df-998c-c8badf5043ea None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:50.864617 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a255bb52-7093-47df-998c-c8badf5043ea None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:50.869336 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97034e3-882e-40c7-9a72-858c529cee39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:50.869962 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97034e3-882e-40c7-9a72-858c529cee39 admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:50.870241 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97034e3-882e-40c7-9a72-858c529cee39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:50.870548 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97034e3-882e-40c7-9a72-858c529cee39 admin admin] PATH_INFO: `/v3/projects` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:50.871192 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97034e3-882e-40c7-9a72-858c529cee39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:50.873129 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97034e3-882e-40c7-9a72-858c529cee39 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:50.951181 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 946/1901] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:19:50 2022] POST /identity/v3/projects => generated 401 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:19:50.955276 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6cabbb-e7f2-49e0-80ea-21d5da614963 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:50.965606 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6cabbb-e7f2-49e0-80ea-21d5da614963 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:50.995536 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:19:51.021619 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d27550-3265-4499-b193-5e60dc042b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:51.022237 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d27550-3265-4499-b193-5e60dc042b53 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:51.022553 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d27550-3265-4499-b193-5e60dc042b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:51.022829 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d27550-3265-4499-b193-5e60dc042b53 admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:51.023468 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d27550-3265-4499-b193-5e60dc042b53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:51.025375 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d27550-3265-4499-b193-5e60dc042b53 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:51.132360 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 956/1902] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:19:50 2022] POST /identity/v3/users => generated 301 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:19:51.136432 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97034e3-882e-40c7-9a72-858c529cee39 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:51.147406 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97034e3-882e-40c7-9a72-858c529cee39 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:51.152098 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c380b3d-73ff-4d1a-b4f1-47ddb4d475b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:51.152641 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c380b3d-73ff-4d1a-b4f1-47ddb4d475b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:51.152924 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c380b3d-73ff-4d1a-b4f1-47ddb4d475b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:51.153227 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c380b3d-73ff-4d1a-b4f1-47ddb4d475b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:51.153849 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c380b3d-73ff-4d1a-b4f1-47ddb4d475b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:51.155512 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c380b3d-73ff-4d1a-b4f1-47ddb4d475b1 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:51.165821 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 947/1903] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:19:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:19:51.169885 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d27550-3265-4499-b193-5e60dc042b53 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:51.180260 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d27550-3265-4499-b193-5e60dc042b53 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:51.185338 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f54852-c32a-40ac-ad07-068ff0a45573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:51.185935 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f54852-c32a-40ac-ad07-068ff0a45573 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:51.186221 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f54852-c32a-40ac-ad07-068ff0a45573 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:51.186536 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f54852-c32a-40ac-ad07-068ff0a45573 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:51.187334 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f54852-c32a-40ac-ad07-068ff0a45573 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:51.188746 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f54852-c32a-40ac-ad07-068ff0a45573 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:51.198553 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 957/1904] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:19:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:19:51.202818 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c380b3d-73ff-4d1a-b4f1-47ddb4d475b1 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:51.212665 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c380b3d-73ff-4d1a-b4f1-47ddb4d475b1 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:51.217184 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b727d802-a2b7-4306-acad-fc3bdcafc94f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:51.217828 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b727d802-a2b7-4306-acad-fc3bdcafc94f admin admin] REQUEST_METHOD: `PUT` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:51.218113 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b727d802-a2b7-4306-acad-fc3bdcafc94f admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:51.218427 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b727d802-a2b7-4306-acad-fc3bdcafc94f admin admin] PATH_INFO: `/v3/projects/130bfeb2c52b4caaaac9b01c565a3afb/users/fca51c4680ee4580a94e27bbd292318b/roles/3255f77192524a16bc405173106871fc` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:51.233093 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b727d802-a2b7-4306-acad-fc3bdcafc94f admin admin] RBAC: Authorizing `identity:create_grant(project_id=130bfeb2c52b4caaaac9b01c565a3afb, user_id=fca51c4680ee4580a94e27bbd292318b, role_id=3255f77192524a16bc405173106871fc)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:51.236396 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b727d802-a2b7-4306-acad-fc3bdcafc94f admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:51.326772 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 948/1905] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:19:51 2022] PUT /identity/v3/projects/130bfeb2c52b4caaaac9b01c565a3afb/users/fca51c4680ee4580a94e27bbd292318b/roles/3255f77192524a16bc405173106871fc => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:19:51.338158 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c4a8fd-aae7-4bc8-823d-9592732b153a None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:51.338484 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c4a8fd-aae7-4bc8-823d-9592732b153a None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:51.338767 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c4a8fd-aae7-4bc8-823d-9592732b153a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:51.374417 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-f7c4a8fd-aae7-4bc8-823d-9592732b153a None None] MFA Rules not processed for user `fca51c4680ee4580a94e27bbd292318b`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:19:51.375636 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-f7c4a8fd-aae7-4bc8-823d-9592732b153a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:51.498465 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b727d802-a2b7-4306-acad-fc3bdcafc94f admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:51.499328 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 958/1906] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:19:51 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:19:51.575138 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b727d802-a2b7-4306-acad-fc3bdcafc94f admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:51.580650 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb13e6e-9d2b-46f6-b769-0243fe6ef9e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:51.581397 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb13e6e-9d2b-46f6-b769-0243fe6ef9e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:51.581765 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb13e6e-9d2b-46f6-b769-0243fe6ef9e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:51.582096 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb13e6e-9d2b-46f6-b769-0243fe6ef9e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:51.582918 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb13e6e-9d2b-46f6-b769-0243fe6ef9e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:51.584942 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb13e6e-9d2b-46f6-b769-0243fe6ef9e8 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:51.690388 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 949/1907] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:19:51 2022] POST /identity/v3/projects => generated 401 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:19:51.691902 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c4a8fd-aae7-4bc8-823d-9592732b153a None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:51.703670 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c4a8fd-aae7-4bc8-823d-9592732b153a None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:51.760718 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769376c7-4208-4a31-b8a9-72f47e7c4194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:51.762023 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769376c7-4208-4a31-b8a9-72f47e7c4194 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:51.762835 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769376c7-4208-4a31-b8a9-72f47e7c4194 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:51.763189 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769376c7-4208-4a31-b8a9-72f47e7c4194 admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:51.764356 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769376c7-4208-4a31-b8a9-72f47e7c4194 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:51.768163 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769376c7-4208-4a31-b8a9-72f47e7c4194 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:51.871015 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 959/1908] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:19:51 2022] POST /identity/v3/users => generated 307 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:19:51.873035 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb13e6e-9d2b-46f6-b769-0243fe6ef9e8 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:51.883854 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb13e6e-9d2b-46f6-b769-0243fe6ef9e8 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:51.888684 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9f4866-8c8c-4ad4-901c-2a9e76e1fa1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:51.889337 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9f4866-8c8c-4ad4-901c-2a9e76e1fa1c admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:51.889622 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9f4866-8c8c-4ad4-901c-2a9e76e1fa1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:51.889923 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9f4866-8c8c-4ad4-901c-2a9e76e1fa1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:51.890621 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9f4866-8c8c-4ad4-901c-2a9e76e1fa1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:51.892780 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9f4866-8c8c-4ad4-901c-2a9e76e1fa1c admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:51.903702 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 950/1909] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:19:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:19:51.908383 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769376c7-4208-4a31-b8a9-72f47e7c4194 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:51.918603 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769376c7-4208-4a31-b8a9-72f47e7c4194 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:51.923864 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973ce50c-54f9-4011-a5b0-1ef7b0f2c0c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:51.924536 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ce50c-54f9-4011-a5b0-1ef7b0f2c0c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:51.924816 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ce50c-54f9-4011-a5b0-1ef7b0f2c0c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:51.925193 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ce50c-54f9-4011-a5b0-1ef7b0f2c0c5 admin admin] PATH_INFO: `/v3/projects/612b0f3a42864fd58919a2d49396d334/users/177e443e234a4f8d834cee618dec014c/roles/62f19ea272b447ccaab1576cd9bf4fd9` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:51.940780 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973ce50c-54f9-4011-a5b0-1ef7b0f2c0c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=612b0f3a42864fd58919a2d49396d334, user_id=177e443e234a4f8d834cee618dec014c, role_id=62f19ea272b447ccaab1576cd9bf4fd9)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:51.943465 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973ce50c-54f9-4011-a5b0-1ef7b0f2c0c5 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:52.038900 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 960/1910] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:19:51 2022] PUT /identity/v3/projects/612b0f3a42864fd58919a2d49396d334/users/177e443e234a4f8d834cee618dec014c/roles/62f19ea272b447ccaab1576cd9bf4fd9 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:19:52.042992 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9f4866-8c8c-4ad4-901c-2a9e76e1fa1c admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:52.059387 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9f4866-8c8c-4ad4-901c-2a9e76e1fa1c admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:52.133801 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e911e4bc-9e69-43b4-9e60-fc58a193cbec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:52.134624 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e911e4bc-9e69-43b4-9e60-fc58a193cbec admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:52.134962 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e911e4bc-9e69-43b4-9e60-fc58a193cbec admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:52.135310 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e911e4bc-9e69-43b4-9e60-fc58a193cbec admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:52.136045 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e911e4bc-9e69-43b4-9e60-fc58a193cbec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:52.137716 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e911e4bc-9e69-43b4-9e60-fc58a193cbec admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:52.147895 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 951/1911] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:19:52 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:19:52.152177 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973ce50c-54f9-4011-a5b0-1ef7b0f2c0c5 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:52.162996 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973ce50c-54f9-4011-a5b0-1ef7b0f2c0c5 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:52.167856 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548c0a67-cbdd-454e-bbfe-0a41cc7e6005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:52.168510 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548c0a67-cbdd-454e-bbfe-0a41cc7e6005 admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:52.175292 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548c0a67-cbdd-454e-bbfe-0a41cc7e6005 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:52.175292 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548c0a67-cbdd-454e-bbfe-0a41cc7e6005 admin admin] PATH_INFO: `/v3/projects/612b0f3a42864fd58919a2d49396d334/users/177e443e234a4f8d834cee618dec014c/roles/62f19ea272b447ccaab1576cd9bf4fd9` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:52.175292 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548c0a67-cbdd-454e-bbfe-0a41cc7e6005 admin admin] RBAC: Authorizing `identity:create_grant(project_id=612b0f3a42864fd58919a2d49396d334, user_id=177e443e234a4f8d834cee618dec014c, role_id=62f19ea272b447ccaab1576cd9bf4fd9)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:52.175292 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548c0a67-cbdd-454e-bbfe-0a41cc7e6005 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:52.176490 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:19:52.185208 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 961/1912] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:19:52 2022] PUT /identity/v3/projects/612b0f3a42864fd58919a2d49396d334/users/177e443e234a4f8d834cee618dec014c/roles/62f19ea272b447ccaab1576cd9bf4fd9 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:19:52.196690 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc71fb0e-0269-49cf-9abb-48a8ef2814d2 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:52.196985 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc71fb0e-0269-49cf-9abb-48a8ef2814d2 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:52.197271 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc71fb0e-0269-49cf-9abb-48a8ef2814d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:52.205232 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:19:52.229660 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-bc71fb0e-0269-49cf-9abb-48a8ef2814d2 None None] MFA Rules not processed for user `177e443e234a4f8d834cee618dec014c`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:19:52.231031 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-bc71fb0e-0269-49cf-9abb-48a8ef2814d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:52.305883 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 952/1913] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:19:52 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:19:52.482849 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548c0a67-cbdd-454e-bbfe-0a41cc7e6005 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:52.486518 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-548c0a67-cbdd-454e-bbfe-0a41cc7e6005 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:52.546366 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548c0a67-cbdd-454e-bbfe-0a41cc7e6005 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:52.550235 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f306c554-9fc3-4d1e-a7d2-42652e3c7042 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60193ba9e1c14df3bc9eebcc6f28f2d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:52.551054 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f306c554-9fc3-4d1e-a7d2-42652e3c7042 service cinder] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:52.551338 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f306c554-9fc3-4d1e-a7d2-42652e3c7042 service cinder] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:52.551642 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f306c554-9fc3-4d1e-a7d2-42652e3c7042 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:52.568696 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f306c554-9fc3-4d1e-a7d2-42652e3c7042 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:52.570351 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f306c554-9fc3-4d1e-a7d2-42652e3c7042 service cinder] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:52.582597 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 962/1914] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:19:52 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:19:52.828870 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc71fb0e-0269-49cf-9abb-48a8ef2814d2 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:52.830934 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-bc71fb0e-0269-49cf-9abb-48a8ef2814d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:52.886140 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f306c554-9fc3-4d1e-a7d2-42652e3c7042 service cinder] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:52.887699 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-f306c554-9fc3-4d1e-a7d2-42652e3c7042 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:52.911045 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f306c554-9fc3-4d1e-a7d2-42652e3c7042 service cinder] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:52.922825 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc71fb0e-0269-49cf-9abb-48a8ef2814d2 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:52.924943 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd73ecb-1941-4b8a-a956-b5ae95d817d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60193ba9e1c14df3bc9eebcc6f28f2d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:52.925675 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd73ecb-1941-4b8a-a956-b5ae95d817d3 service cinder] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:52.925856 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd73ecb-1941-4b8a-a956-b5ae95d817d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:52.926675 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd73ecb-1941-4b8a-a956-b5ae95d817d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:52.932074 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7ca79b-efbd-49ad-9297-c8bc0580c1e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:52.932773 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7ca79b-efbd-49ad-9297-c8bc0580c1e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:52.933058 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7ca79b-efbd-49ad-9297-c8bc0580c1e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:52.933333 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7ca79b-efbd-49ad-9297-c8bc0580c1e7 admin admin] PATH_INFO: `/v3/users/a826cf4a5ac14d0d9714d37c2b58444f` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:52.935517 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7ca79b-efbd-49ad-9297-c8bc0580c1e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a826cf4a5ac14d0d9714d37c2b58444f)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:52.937698 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7ca79b-efbd-49ad-9297-c8bc0580c1e7 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:53.018181 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd73ecb-1941-4b8a-a956-b5ae95d817d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:53.019812 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd73ecb-1941-4b8a-a956-b5ae95d817d3 service cinder] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:53.031498 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 963/1915] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:19:52 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 25 03:19:53.032529 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4451422d-56e5-45b7-b105-a731ea2f90d9 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:53.032857 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4451422d-56e5-45b7-b105-a731ea2f90d9 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:53.033136 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4451422d-56e5-45b7-b105-a731ea2f90d9 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:53.040025 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 964/1916] 10.222.0.58 () {54 vars in 921 bytes} [Thu Aug 25 03:19:53 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:19:53.042019 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db4ab1e-2970-4a95-99d7-f0f6cb7cd800 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:53.042331 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db4ab1e-2970-4a95-99d7-f0f6cb7cd800 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:53.042614 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db4ab1e-2970-4a95-99d7-f0f6cb7cd800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:53.083119 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-3db4ab1e-2970-4a95-99d7-f0f6cb7cd800 None None] MFA Rules not processed for user `0cd171bf4df94f8fab397e42835feeab`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:19:53.084408 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-3db4ab1e-2970-4a95-99d7-f0f6cb7cd800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:53.201021 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-8f7ca79b-efbd-49ad-9297-c8bc0580c1e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a826cf4a5ac14d0d9714d37c2b58444f'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:53.277124 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 965/1917] 10.222.0.58 () {58 vars in 1018 bytes} [Thu Aug 25 03:19:53 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:19:53.309134 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-8f7ca79b-efbd-49ad-9297-c8bc0580c1e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a826cf4a5ac14d0d9714d37c2b58444f'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:53.309958 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-8f7ca79b-efbd-49ad-9297-c8bc0580c1e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a826cf4a5ac14d0d9714d37c2b58444f'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:53.310722 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-8f7ca79b-efbd-49ad-9297-c8bc0580c1e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a826cf4a5ac14d0d9714d37c2b58444f'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:53.327614 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-8f7ca79b-efbd-49ad-9297-c8bc0580c1e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a826cf4a5ac14d0d9714d37c2b58444f'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:53.350189 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 953/1918] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:19:52 2022] DELETE /identity/v3/users/a826cf4a5ac14d0d9714d37c2b58444f => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:19:53.354622 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cae03c5-69f2-4ba9-b60a-a2fd7e1d203b None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:53.354983 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cae03c5-69f2-4ba9-b60a-a2fd7e1d203b None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:53.355364 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cae03c5-69f2-4ba9-b60a-a2fd7e1d203b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:53.394521 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-1cae03c5-69f2-4ba9-b60a-a2fd7e1d203b None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:19:53.395907 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-1cae03c5-69f2-4ba9-b60a-a2fd7e1d203b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:53.433617 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:19:53.494231 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 966/1919] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:19:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:19:53.501990 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7ca79b-efbd-49ad-9297-c8bc0580c1e7 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:53.503684 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-8f7ca79b-efbd-49ad-9297-c8bc0580c1e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:53.519785 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:19:53.613025 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7ca79b-efbd-49ad-9297-c8bc0580c1e7 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:53.617250 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfaf9b8-c9fd-4050-abca-b1fd7603ad7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:53.618034 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfaf9b8-c9fd-4050-abca-b1fd7603ad7a service neutron] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:53.618365 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfaf9b8-c9fd-4050-abca-b1fd7603ad7a service neutron] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:53.619201 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfaf9b8-c9fd-4050-abca-b1fd7603ad7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:53.621522 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cae03c5-69f2-4ba9-b60a-a2fd7e1d203b None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:53.623481 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-1cae03c5-69f2-4ba9-b60a-a2fd7e1d203b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:53.638182 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfaf9b8-c9fd-4050-abca-b1fd7603ad7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:53.639810 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfaf9b8-c9fd-4050-abca-b1fd7603ad7a service neutron] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:53.655114 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 954/1920] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:19:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:19:53.722908 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cae03c5-69f2-4ba9-b60a-a2fd7e1d203b None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:53.728798 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16adc350-0a62-4288-9b1b-7a925201b9d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cd171bf4df94f8fab397e42835feeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:53.729595 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16adc350-0a62-4288-9b1b-7a925201b9d2 service nova] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:53.729882 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16adc350-0a62-4288-9b1b-7a925201b9d2 service nova] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:53.730158 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16adc350-0a62-4288-9b1b-7a925201b9d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:53.731998 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-16adc350-0a62-4288-9b1b-7a925201b9d2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:53.745907 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16adc350-0a62-4288-9b1b-7a925201b9d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:53.747065 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16adc350-0a62-4288-9b1b-7a925201b9d2 service nova] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:53.771760 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 967/1921] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:19:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:19:54.179496 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfaf9b8-c9fd-4050-abca-b1fd7603ad7a service neutron] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:54.181261 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-4bfaf9b8-c9fd-4050-abca-b1fd7603ad7a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:54.198712 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfaf9b8-c9fd-4050-abca-b1fd7603ad7a service neutron] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:54.204906 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9cb814-92f3-4a13-8cf4-6b44d475e908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:54.206115 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9cb814-92f3-4a13-8cf4-6b44d475e908 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:54.206618 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9cb814-92f3-4a13-8cf4-6b44d475e908 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:54.207090 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9cb814-92f3-4a13-8cf4-6b44d475e908 admin admin] PATH_INFO: `/v3/projects/de1fb19b5e3e4e03acb2f6be518a8a82` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:54.209298 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9cb814-92f3-4a13-8cf4-6b44d475e908 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de1fb19b5e3e4e03acb2f6be518a8a82)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:54.212611 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9cb814-92f3-4a13-8cf4-6b44d475e908 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:54.351030 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-6c9cb814-92f3-4a13-8cf4-6b44d475e908 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de1fb19b5e3e4e03acb2f6be518a8a82', 'request_id': 'req-6c9cb814-92f3-4a13-8cf4-6b44d475e908'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:54.447393 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-6c9cb814-92f3-4a13-8cf4-6b44d475e908 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de1fb19b5e3e4e03acb2f6be518a8a82', 'request_id': 'req-6c9cb814-92f3-4a13-8cf4-6b44d475e908'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:54.456277 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-6c9cb814-92f3-4a13-8cf4-6b44d475e908 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de1fb19b5e3e4e03acb2f6be518a8a82', 'request_id': 'req-6c9cb814-92f3-4a13-8cf4-6b44d475e908'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:54.480889 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-6c9cb814-92f3-4a13-8cf4-6b44d475e908 admin admin] The token cache is being invalidate because project de1fb19b5e3e4e03acb2f6be518a8a82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81995) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:19:54.481190 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-6c9cb814-92f3-4a13-8cf4-6b44d475e908 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:54.481880 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-6c9cb814-92f3-4a13-8cf4-6b44d475e908 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:54.483426 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 955/1922] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:19:54 2022] DELETE /identity/v3/projects/de1fb19b5e3e4e03acb2f6be518a8a82 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:19:54.487555 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16adc350-0a62-4288-9b1b-7a925201b9d2 service nova] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:54.506308 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16adc350-0a62-4288-9b1b-7a925201b9d2 service nova] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:54.581709 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68cae56-e5e5-457e-801a-2808615835cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:54.582419 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68cae56-e5e5-457e-801a-2808615835cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:54.582702 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68cae56-e5e5-457e-801a-2808615835cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:54.582979 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68cae56-e5e5-457e-801a-2808615835cd admin admin] PATH_INFO: `/v3/users/03619407217c40c898ff3b2cdddc6416` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:54.587937 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68cae56-e5e5-457e-801a-2808615835cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=03619407217c40c898ff3b2cdddc6416)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:54.589907 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68cae56-e5e5-457e-801a-2808615835cd admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:54.792846 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-c68cae56-e5e5-457e-801a-2808615835cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03619407217c40c898ff3b2cdddc6416'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:54.860604 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-c68cae56-e5e5-457e-801a-2808615835cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03619407217c40c898ff3b2cdddc6416'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:54.864708 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-c68cae56-e5e5-457e-801a-2808615835cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03619407217c40c898ff3b2cdddc6416'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:54.872586 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:19:54.889141 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-c68cae56-e5e5-457e-801a-2808615835cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03619407217c40c898ff3b2cdddc6416'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:54.902905 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-c68cae56-e5e5-457e-801a-2808615835cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03619407217c40c898ff3b2cdddc6416'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:54.909896 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 968/1923] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:19:54 2022] DELETE /identity/v3/users/03619407217c40c898ff3b2cdddc6416 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:19:55.381009 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9cb814-92f3-4a13-8cf4-6b44d475e908 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:55.383769 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-6c9cb814-92f3-4a13-8cf4-6b44d475e908 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:55.415871 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9cb814-92f3-4a13-8cf4-6b44d475e908 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:55.482156 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd088c0-2ffa-4388-9851-2f275ec22326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:55.482846 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd088c0-2ffa-4388-9851-2f275ec22326 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:55.483124 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd088c0-2ffa-4388-9851-2f275ec22326 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:55.483404 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd088c0-2ffa-4388-9851-2f275ec22326 admin admin] PATH_INFO: `/v3/projects/3530c02d0d5a4e4ea6cdf9ec23fa12e6` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:55.484626 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd088c0-2ffa-4388-9851-2f275ec22326 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3530c02d0d5a4e4ea6cdf9ec23fa12e6)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:55.486786 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd088c0-2ffa-4388-9851-2f275ec22326 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:55.832635 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:19:55.911145 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-9fd088c0-2ffa-4388-9851-2f275ec22326 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3530c02d0d5a4e4ea6cdf9ec23fa12e6', 'request_id': 'req-9fd088c0-2ffa-4388-9851-2f275ec22326'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:56.034636 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-9fd088c0-2ffa-4388-9851-2f275ec22326 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3530c02d0d5a4e4ea6cdf9ec23fa12e6', 'request_id': 'req-9fd088c0-2ffa-4388-9851-2f275ec22326'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:56.040889 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-9fd088c0-2ffa-4388-9851-2f275ec22326 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3530c02d0d5a4e4ea6cdf9ec23fa12e6', 'request_id': 'req-9fd088c0-2ffa-4388-9851-2f275ec22326'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:56.052851 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-9fd088c0-2ffa-4388-9851-2f275ec22326 admin admin] The token cache is being invalidate because project 3530c02d0d5a4e4ea6cdf9ec23fa12e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81995) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:19:56.053165 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-9fd088c0-2ffa-4388-9851-2f275ec22326 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:56.053805 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-9fd088c0-2ffa-4388-9851-2f275ec22326 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:56.055348 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 956/1924] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:19:55 2022] DELETE /identity/v3/projects/3530c02d0d5a4e4ea6cdf9ec23fa12e6 => generated 0 bytes in 675 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:19:56.060260 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1d35db-9687-4a79-b3fc-3cb64e80fa9e None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:56.060745 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1d35db-9687-4a79-b3fc-3cb64e80fa9e None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:56.060846 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1d35db-9687-4a79-b3fc-3cb64e80fa9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:56.086859 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-7d1d35db-9687-4a79-b3fc-3cb64e80fa9e None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:19:56.088113 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-7d1d35db-9687-4a79-b3fc-3cb64e80fa9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:56.158840 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 969/1925] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:19:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:19:56.167733 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77efac0b-1bd7-455a-b014-c066263540ea None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:56.168228 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77efac0b-1bd7-455a-b014-c066263540ea None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:56.168352 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77efac0b-1bd7-455a-b014-c066263540ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:56.189746 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-77efac0b-1bd7-455a-b014-c066263540ea None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:19:56.191176 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-77efac0b-1bd7-455a-b014-c066263540ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:56.200017 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 957/1926] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:19:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:19:56.206622 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1d35db-9687-4a79-b3fc-3cb64e80fa9e None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:56.217908 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1d35db-9687-4a79-b3fc-3cb64e80fa9e None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:56.222920 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ae3772-02a3-4e0e-bae9-7ec67ac05c27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:56.223654 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ae3772-02a3-4e0e-bae9-7ec67ac05c27 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:56.223942 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ae3772-02a3-4e0e-bae9-7ec67ac05c27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:56.224220 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ae3772-02a3-4e0e-bae9-7ec67ac05c27 admin admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:56.225338 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ae3772-02a3-4e0e-bae9-7ec67ac05c27 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:19:56.225338 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ae3772-02a3-4e0e-bae9-7ec67ac05c27 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:56.226915 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ae3772-02a3-4e0e-bae9-7ec67ac05c27 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:56.242329 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 970/1927] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Aug 25 03:19:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:19:56.245266 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77efac0b-1bd7-455a-b014-c066263540ea None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:56.272317 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77efac0b-1bd7-455a-b014-c066263540ea None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:56.280296 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedce3bd-07ba-459f-822b-bcb34c1682d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:56.280921 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedce3bd-07ba-459f-822b-bcb34c1682d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:56.281290 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedce3bd-07ba-459f-822b-bcb34c1682d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:56.281559 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedce3bd-07ba-459f-822b-bcb34c1682d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:56.282380 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedce3bd-07ba-459f-822b-bcb34c1682d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:56.284759 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedce3bd-07ba-459f-822b-bcb34c1682d5 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:56.395869 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 958/1928] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:19:56 2022] POST /identity/v3/projects => generated 385 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:19:56.400207 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ae3772-02a3-4e0e-bae9-7ec67ac05c27 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:56.411049 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ae3772-02a3-4e0e-bae9-7ec67ac05c27 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:56.468420 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076d27c7-cb72-4c10-87c3-434a8558733e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:56.469068 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076d27c7-cb72-4c10-87c3-434a8558733e admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:56.469353 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076d27c7-cb72-4c10-87c3-434a8558733e admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:56.469637 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076d27c7-cb72-4c10-87c3-434a8558733e admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:56.470803 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076d27c7-cb72-4c10-87c3-434a8558733e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:56.472838 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076d27c7-cb72-4c10-87c3-434a8558733e admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:56.555371 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 971/1929] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:19:56 2022] POST /identity/v3/users => generated 293 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:19:56.559534 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedce3bd-07ba-459f-822b-bcb34c1682d5 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:56.570498 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedce3bd-07ba-459f-822b-bcb34c1682d5 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:56.575387 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4b6566-417e-4c40-a741-72b2781254c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:56.575982 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4b6566-417e-4c40-a741-72b2781254c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:56.576265 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4b6566-417e-4c40-a741-72b2781254c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:56.576542 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4b6566-417e-4c40-a741-72b2781254c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:56.577186 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4b6566-417e-4c40-a741-72b2781254c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:56.578931 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4b6566-417e-4c40-a741-72b2781254c1 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:56.588916 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 959/1930] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:19:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:19:56.594147 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076d27c7-cb72-4c10-87c3-434a8558733e admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:56.604656 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076d27c7-cb72-4c10-87c3-434a8558733e admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:56.609201 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7a1acb-7b45-4dee-98e2-a6d6b9ec2631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:56.609796 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7a1acb-7b45-4dee-98e2-a6d6b9ec2631 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:56.610073 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7a1acb-7b45-4dee-98e2-a6d6b9ec2631 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:56.610377 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7a1acb-7b45-4dee-98e2-a6d6b9ec2631 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:56.611695 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7a1acb-7b45-4dee-98e2-a6d6b9ec2631 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:56.613276 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7a1acb-7b45-4dee-98e2-a6d6b9ec2631 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:56.623676 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 972/1931] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:19:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:19:56.627805 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4b6566-417e-4c40-a741-72b2781254c1 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:56.639163 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4b6566-417e-4c40-a741-72b2781254c1 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:56.643739 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024d1ad9-fb38-45e8-90ca-25534688d1ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:56.644483 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024d1ad9-fb38-45e8-90ca-25534688d1ff admin admin] REQUEST_METHOD: `PUT` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:56.644761 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024d1ad9-fb38-45e8-90ca-25534688d1ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:56.645036 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024d1ad9-fb38-45e8-90ca-25534688d1ff admin admin] PATH_INFO: `/v3/projects/a5113827072c453580a777ec3a60ab13/users/6c6aa9def0bf4cb6aac3c52282dafe90/roles/3255f77192524a16bc405173106871fc` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:56.659667 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024d1ad9-fb38-45e8-90ca-25534688d1ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5113827072c453580a777ec3a60ab13, user_id=6c6aa9def0bf4cb6aac3c52282dafe90, role_id=3255f77192524a16bc405173106871fc)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:56.662446 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024d1ad9-fb38-45e8-90ca-25534688d1ff admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:56.714340 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 960/1932] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:19:56 2022] PUT /identity/v3/projects/a5113827072c453580a777ec3a60ab13/users/6c6aa9def0bf4cb6aac3c52282dafe90/roles/3255f77192524a16bc405173106871fc => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:19:56.725392 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194bc333-c3b6-4d0b-96ea-c0b72e80d02b None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:56.725682 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194bc333-c3b6-4d0b-96ea-c0b72e80d02b None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:56.725958 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194bc333-c3b6-4d0b-96ea-c0b72e80d02b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:56.752537 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-194bc333-c3b6-4d0b-96ea-c0b72e80d02b None None] MFA Rules not processed for user `6c6aa9def0bf4cb6aac3c52282dafe90`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:19:56.753842 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-194bc333-c3b6-4d0b-96ea-c0b72e80d02b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:56.784061 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:19:56.824404 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 973/1933] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:19:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:19:56.838414 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024d1ad9-fb38-45e8-90ca-25534688d1ff admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:56.840382 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-024d1ad9-fb38-45e8-90ca-25534688d1ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:56.896979 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024d1ad9-fb38-45e8-90ca-25534688d1ff admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:56.900596 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8e2fb4-6360-42ea-818f-cd4daa73dde0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60193ba9e1c14df3bc9eebcc6f28f2d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:56.901231 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8e2fb4-6360-42ea-818f-cd4daa73dde0 service cinder] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:56.901508 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8e2fb4-6360-42ea-818f-cd4daa73dde0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:56.901780 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8e2fb4-6360-42ea-818f-cd4daa73dde0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:56.910985 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8e2fb4-6360-42ea-818f-cd4daa73dde0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:56.912412 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8e2fb4-6360-42ea-818f-cd4daa73dde0 service cinder] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:56.921279 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 961/1934] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:19:56 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:19:56.940345 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194bc333-c3b6-4d0b-96ea-c0b72e80d02b None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:57.001412 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194bc333-c3b6-4d0b-96ea-c0b72e80d02b None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:57.005947 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39c6bea-bbdf-40e9-89c8-37ab12314514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:57.006604 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c6bea-bbdf-40e9-89c8-37ab12314514 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:57.006885 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c6bea-bbdf-40e9-89c8-37ab12314514 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:57.007161 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c6bea-bbdf-40e9-89c8-37ab12314514 admin admin] PATH_INFO: `/v3/users/6c6aa9def0bf4cb6aac3c52282dafe90` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:57.008273 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39c6bea-bbdf-40e9-89c8-37ab12314514 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c6aa9def0bf4cb6aac3c52282dafe90)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:57.010178 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39c6bea-bbdf-40e9-89c8-37ab12314514 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:57.165742 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-a39c6bea-bbdf-40e9-89c8-37ab12314514 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c6aa9def0bf4cb6aac3c52282dafe90'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:57.234717 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-a39c6bea-bbdf-40e9-89c8-37ab12314514 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c6aa9def0bf4cb6aac3c52282dafe90'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:57.235373 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-a39c6bea-bbdf-40e9-89c8-37ab12314514 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c6aa9def0bf4cb6aac3c52282dafe90'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:57.243035 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-a39c6bea-bbdf-40e9-89c8-37ab12314514 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c6aa9def0bf4cb6aac3c52282dafe90'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:57.256346 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-a39c6bea-bbdf-40e9-89c8-37ab12314514 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c6aa9def0bf4cb6aac3c52282dafe90'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:57.262739 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 974/1935] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:19:56 2022] DELETE /identity/v3/users/6c6aa9def0bf4cb6aac3c52282dafe90 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:19:57.268034 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e2cc6e-2df8-483c-bc01-97affd2ca9ee None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:57.268323 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e2cc6e-2df8-483c-bc01-97affd2ca9ee None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:57.268596 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e2cc6e-2df8-483c-bc01-97affd2ca9ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:57.287849 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-95e2cc6e-2df8-483c-bc01-97affd2ca9ee None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:19:57.289109 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-95e2cc6e-2df8-483c-bc01-97affd2ca9ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:57.387543 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 962/1936] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:19:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:19:57.398298 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39c6bea-bbdf-40e9-89c8-37ab12314514 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:57.401293 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-a39c6bea-bbdf-40e9-89c8-37ab12314514 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:57.474110 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:19:57.483827 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39c6bea-bbdf-40e9-89c8-37ab12314514 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:57.489269 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc48f9f9-cd83-477b-a03c-0f934228e076 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:57.490088 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc48f9f9-cd83-477b-a03c-0f934228e076 service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:57.491323 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc48f9f9-cd83-477b-a03c-0f934228e076 service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:57.491752 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc48f9f9-cd83-477b-a03c-0f934228e076 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:57.503927 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc48f9f9-cd83-477b-a03c-0f934228e076 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:57.506073 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc48f9f9-cd83-477b-a03c-0f934228e076 service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:57.521965 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 975/1937] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:19:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:19:57.962762 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e2cc6e-2df8-483c-bc01-97affd2ca9ee None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:57.964400 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-95e2cc6e-2df8-483c-bc01-97affd2ca9ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:57.972771 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:19:57.979023 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e2cc6e-2df8-483c-bc01-97affd2ca9ee None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:57.984363 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dff21a-c40a-4731-876f-41135596e849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:57.985024 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dff21a-c40a-4731-876f-41135596e849 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:57.985306 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dff21a-c40a-4731-876f-41135596e849 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:57.985579 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dff21a-c40a-4731-876f-41135596e849 admin admin] PATH_INFO: `/v3/projects/a5113827072c453580a777ec3a60ab13` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:57.987440 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dff21a-c40a-4731-876f-41135596e849 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5113827072c453580a777ec3a60ab13)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:57.989558 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dff21a-c40a-4731-876f-41135596e849 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:58.116310 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-d8dff21a-c40a-4731-876f-41135596e849 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5113827072c453580a777ec3a60ab13', 'request_id': 'req-d8dff21a-c40a-4731-876f-41135596e849'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:58.200742 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-d8dff21a-c40a-4731-876f-41135596e849 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5113827072c453580a777ec3a60ab13', 'request_id': 'req-d8dff21a-c40a-4731-876f-41135596e849'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:58.212849 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-d8dff21a-c40a-4731-876f-41135596e849 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5113827072c453580a777ec3a60ab13', 'request_id': 'req-d8dff21a-c40a-4731-876f-41135596e849'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:58.219031 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-d8dff21a-c40a-4731-876f-41135596e849 admin admin] The token cache is being invalidate because project a5113827072c453580a777ec3a60ab13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81995) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:19:58.219318 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-d8dff21a-c40a-4731-876f-41135596e849 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:58.219965 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-d8dff21a-c40a-4731-876f-41135596e849 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:58.222139 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 963/1938] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:19:57 2022] DELETE /identity/v3/projects/a5113827072c453580a777ec3a60ab13 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:19:58.227153 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4473e5-4c97-4c0b-901c-2e6298500bf3 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:58.227653 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4473e5-4c97-4c0b-901c-2e6298500bf3 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:58.227768 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4473e5-4c97-4c0b-901c-2e6298500bf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:58.246697 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-6f4473e5-4c97-4c0b-901c-2e6298500bf3 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:19:58.247904 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-6f4473e5-4c97-4c0b-901c-2e6298500bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:58.307700 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 976/1939] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:19:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:19:58.316886 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302d2d23-ab1d-443d-8836-3ad7f22a17dd None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:58.317176 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302d2d23-ab1d-443d-8836-3ad7f22a17dd None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:58.317512 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302d2d23-ab1d-443d-8836-3ad7f22a17dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:58.340870 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-302d2d23-ab1d-443d-8836-3ad7f22a17dd None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:19:58.342419 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-302d2d23-ab1d-443d-8836-3ad7f22a17dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:58.353174 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 964/1940] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:19:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:19:58.358208 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4473e5-4c97-4c0b-901c-2e6298500bf3 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:58.368670 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4473e5-4c97-4c0b-901c-2e6298500bf3 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:58.376260 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064b8378-77a1-43be-a48a-7c186d881f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:58.377038 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064b8378-77a1-43be-a48a-7c186d881f68 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:58.377474 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064b8378-77a1-43be-a48a-7c186d881f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:58.377832 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064b8378-77a1-43be-a48a-7c186d881f68 admin admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:58.378786 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064b8378-77a1-43be-a48a-7c186d881f68 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:19:58.379108 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064b8378-77a1-43be-a48a-7c186d881f68 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:58.380835 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064b8378-77a1-43be-a48a-7c186d881f68 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:58.398998 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 977/1941] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Aug 25 03:19:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:19:58.402484 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302d2d23-ab1d-443d-8836-3ad7f22a17dd None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:58.412630 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302d2d23-ab1d-443d-8836-3ad7f22a17dd None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:58.424133 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed936d80-9928-48dd-8ba4-86003ac532c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:58.424766 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed936d80-9928-48dd-8ba4-86003ac532c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:58.426543 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed936d80-9928-48dd-8ba4-86003ac532c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:58.426854 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed936d80-9928-48dd-8ba4-86003ac532c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:58.427524 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed936d80-9928-48dd-8ba4-86003ac532c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:58.429564 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed936d80-9928-48dd-8ba4-86003ac532c5 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:58.549672 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 965/1942] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:19:58 2022] POST /identity/v3/projects => generated 371 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:19:58.552010 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064b8378-77a1-43be-a48a-7c186d881f68 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:58.565002 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064b8378-77a1-43be-a48a-7c186d881f68 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:58.646138 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c62e8b6-fa07-40a5-b1fc-0a60ba6b64d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:58.646841 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c62e8b6-fa07-40a5-b1fc-0a60ba6b64d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:58.647145 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c62e8b6-fa07-40a5-b1fc-0a60ba6b64d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:58.647424 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c62e8b6-fa07-40a5-b1fc-0a60ba6b64d0 admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:58.648197 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c62e8b6-fa07-40a5-b1fc-0a60ba6b64d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:58.650213 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c62e8b6-fa07-40a5-b1fc-0a60ba6b64d0 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:58.755209 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 978/1943] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:19:58 2022] POST /identity/v3/users => generated 286 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:19:58.759230 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed936d80-9928-48dd-8ba4-86003ac532c5 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:58.769822 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed936d80-9928-48dd-8ba4-86003ac532c5 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:58.774401 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9904ef60-3fad-4718-b337-40530583a389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:58.774998 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9904ef60-3fad-4718-b337-40530583a389 admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:58.775280 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9904ef60-3fad-4718-b337-40530583a389 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:58.775597 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9904ef60-3fad-4718-b337-40530583a389 admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:58.776193 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9904ef60-3fad-4718-b337-40530583a389 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:58.777785 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9904ef60-3fad-4718-b337-40530583a389 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:58.787780 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 966/1944] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:19:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:19:58.791567 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c62e8b6-fa07-40a5-b1fc-0a60ba6b64d0 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:58.801827 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c62e8b6-fa07-40a5-b1fc-0a60ba6b64d0 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:58.806605 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16648aef-ad77-45f3-9b4c-963524fd651e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:58.807211 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16648aef-ad77-45f3-9b4c-963524fd651e admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:58.807497 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16648aef-ad77-45f3-9b4c-963524fd651e admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:58.807773 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16648aef-ad77-45f3-9b4c-963524fd651e admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:58.808448 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16648aef-ad77-45f3-9b4c-963524fd651e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:58.810074 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16648aef-ad77-45f3-9b4c-963524fd651e admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:58.821274 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 979/1945] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:19:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:19:58.825723 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9904ef60-3fad-4718-b337-40530583a389 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:58.836254 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9904ef60-3fad-4718-b337-40530583a389 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:58.840757 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0632c237-e422-4786-9583-9bac1c35c971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:58.841410 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0632c237-e422-4786-9583-9bac1c35c971 admin admin] REQUEST_METHOD: `PUT` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:58.841694 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0632c237-e422-4786-9583-9bac1c35c971 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:58.841975 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0632c237-e422-4786-9583-9bac1c35c971 admin admin] PATH_INFO: `/v3/projects/d34a479ddd544089b02e4937c150e9f0/users/700cf9e7f6184c11a4f718d35c0b14fd/roles/3255f77192524a16bc405173106871fc` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:58.856546 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0632c237-e422-4786-9583-9bac1c35c971 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d34a479ddd544089b02e4937c150e9f0, user_id=700cf9e7f6184c11a4f718d35c0b14fd, role_id=3255f77192524a16bc405173106871fc)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:58.859242 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0632c237-e422-4786-9583-9bac1c35c971 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:58.908250 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 967/1946] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:19:58 2022] PUT /identity/v3/projects/d34a479ddd544089b02e4937c150e9f0/users/700cf9e7f6184c11a4f718d35c0b14fd/roles/3255f77192524a16bc405173106871fc => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:19:58.918700 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e689ed9-48e7-4488-b391-3be42b1609e5 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:58.919009 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e689ed9-48e7-4488-b391-3be42b1609e5 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:58.919283 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e689ed9-48e7-4488-b391-3be42b1609e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:58.947288 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-4e689ed9-48e7-4488-b391-3be42b1609e5 None None] MFA Rules not processed for user `700cf9e7f6184c11a4f718d35c0b14fd`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:19:58.948438 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-4e689ed9-48e7-4488-b391-3be42b1609e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:58.990762 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:19:59.010818 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 980/1947] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:19:58 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:19:59.022556 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0632c237-e422-4786-9583-9bac1c35c971 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:59.024360 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-0632c237-e422-4786-9583-9bac1c35c971 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:59.079082 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0632c237-e422-4786-9583-9bac1c35c971 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:59.083132 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87afa1ea-b359-4495-88b6-22ac0826dd24 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60193ba9e1c14df3bc9eebcc6f28f2d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:59.083729 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87afa1ea-b359-4495-88b6-22ac0826dd24 service cinder] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:59.084005 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87afa1ea-b359-4495-88b6-22ac0826dd24 service cinder] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:59.084277 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87afa1ea-b359-4495-88b6-22ac0826dd24 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:59.092153 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87afa1ea-b359-4495-88b6-22ac0826dd24 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:59.093557 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87afa1ea-b359-4495-88b6-22ac0826dd24 service cinder] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:59.102815 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 968/1948] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:19:59 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:19:59.123677 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e689ed9-48e7-4488-b391-3be42b1609e5 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:59.184601 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e689ed9-48e7-4488-b391-3be42b1609e5 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:59.189135 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819ff1b4-a922-4ae1-b90e-cca5e6430a07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:59.189761 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819ff1b4-a922-4ae1-b90e-cca5e6430a07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:59.190041 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819ff1b4-a922-4ae1-b90e-cca5e6430a07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:59.190345 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819ff1b4-a922-4ae1-b90e-cca5e6430a07 admin admin] PATH_INFO: `/v3/users/700cf9e7f6184c11a4f718d35c0b14fd` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:59.191477 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819ff1b4-a922-4ae1-b90e-cca5e6430a07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=700cf9e7f6184c11a4f718d35c0b14fd)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:59.193391 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819ff1b4-a922-4ae1-b90e-cca5e6430a07 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:59.380690 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-819ff1b4-a922-4ae1-b90e-cca5e6430a07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '700cf9e7f6184c11a4f718d35c0b14fd'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:59.434724 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-819ff1b4-a922-4ae1-b90e-cca5e6430a07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '700cf9e7f6184c11a4f718d35c0b14fd'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:59.435370 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-819ff1b4-a922-4ae1-b90e-cca5e6430a07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '700cf9e7f6184c11a4f718d35c0b14fd'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:59.443617 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-819ff1b4-a922-4ae1-b90e-cca5e6430a07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '700cf9e7f6184c11a4f718d35c0b14fd'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:59.456547 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-819ff1b4-a922-4ae1-b90e-cca5e6430a07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '700cf9e7f6184c11a4f718d35c0b14fd'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:19:59.462510 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 981/1949] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:19:59 2022] DELETE /identity/v3/users/700cf9e7f6184c11a4f718d35c0b14fd => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:19:59.466913 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9181719-869a-4100-8b00-a9edcb20f561 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:59.467235 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9181719-869a-4100-8b00-a9edcb20f561 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:59.467646 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9181719-869a-4100-8b00-a9edcb20f561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:59.486509 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-f9181719-869a-4100-8b00-a9edcb20f561 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:19:59.494584 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-f9181719-869a-4100-8b00-a9edcb20f561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:59.507024 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:19:59.548356 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 969/1950] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:19:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:19:59.558309 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819ff1b4-a922-4ae1-b90e-cca5e6430a07 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:59.559837 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-819ff1b4-a922-4ae1-b90e-cca5e6430a07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:59.628497 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819ff1b4-a922-4ae1-b90e-cca5e6430a07 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:59.632198 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f71bf8-c690-4e69-b651-270e7dcf7e95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:59.632816 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f71bf8-c690-4e69-b651-270e7dcf7e95 service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:59.633154 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f71bf8-c690-4e69-b651-270e7dcf7e95 service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:59.633439 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f71bf8-c690-4e69-b651-270e7dcf7e95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:59.643812 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f71bf8-c690-4e69-b651-270e7dcf7e95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:59.645354 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f71bf8-c690-4e69-b651-270e7dcf7e95 service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:59.655225 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 982/1951] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:19:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:20:00.195878 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9181719-869a-4100-8b00-a9edcb20f561 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:00.204771 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-f9181719-869a-4100-8b00-a9edcb20f561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:00.211684 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9181719-869a-4100-8b00-a9edcb20f561 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:00.217162 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd57cc2-14c8-4e35-92fc-53c2ff1fe0b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:00.217800 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd57cc2-14c8-4e35-92fc-53c2ff1fe0b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:00.218083 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd57cc2-14c8-4e35-92fc-53c2ff1fe0b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:00.218414 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd57cc2-14c8-4e35-92fc-53c2ff1fe0b0 admin admin] PATH_INFO: `/v3/projects/d34a479ddd544089b02e4937c150e9f0` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:00.219539 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd57cc2-14c8-4e35-92fc-53c2ff1fe0b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d34a479ddd544089b02e4937c150e9f0)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:00.221986 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd57cc2-14c8-4e35-92fc-53c2ff1fe0b0 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:00.366896 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-4cd57cc2-14c8-4e35-92fc-53c2ff1fe0b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd34a479ddd544089b02e4937c150e9f0', 'request_id': 'req-4cd57cc2-14c8-4e35-92fc-53c2ff1fe0b0'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:00.457508 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-4cd57cc2-14c8-4e35-92fc-53c2ff1fe0b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd34a479ddd544089b02e4937c150e9f0', 'request_id': 'req-4cd57cc2-14c8-4e35-92fc-53c2ff1fe0b0'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:00.463896 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-4cd57cc2-14c8-4e35-92fc-53c2ff1fe0b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd34a479ddd544089b02e4937c150e9f0', 'request_id': 'req-4cd57cc2-14c8-4e35-92fc-53c2ff1fe0b0'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:00.476731 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-4cd57cc2-14c8-4e35-92fc-53c2ff1fe0b0 admin admin] The token cache is being invalidate because project d34a479ddd544089b02e4937c150e9f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81995) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:20:00.477023 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-4cd57cc2-14c8-4e35-92fc-53c2ff1fe0b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:00.477667 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-4cd57cc2-14c8-4e35-92fc-53c2ff1fe0b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:00.479185 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 970/1952] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:20:00 2022] DELETE /identity/v3/projects/d34a479ddd544089b02e4937c150e9f0 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:20:00.484084 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc3f906-4498-4325-bfb8-67c541ce92b5 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:00.484624 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc3f906-4498-4325-bfb8-67c541ce92b5 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:00.484907 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc3f906-4498-4325-bfb8-67c541ce92b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:00.498794 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:20:00.504732 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-5cc3f906-4498-4325-bfb8-67c541ce92b5 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:20:00.505982 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-5cc3f906-4498-4325-bfb8-67c541ce92b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:00.564119 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 983/1953] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:20:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:20:00.572318 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62e69b8-f527-4312-a8f6-da0f855d1ada None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:00.572768 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62e69b8-f527-4312-a8f6-da0f855d1ada None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:00.573130 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62e69b8-f527-4312-a8f6-da0f855d1ada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:00.594954 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-c62e69b8-f527-4312-a8f6-da0f855d1ada None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:20:00.596719 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-c62e69b8-f527-4312-a8f6-da0f855d1ada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:00.606332 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 971/1954] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:20:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:20:00.610657 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc3f906-4498-4325-bfb8-67c541ce92b5 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:00.620306 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc3f906-4498-4325-bfb8-67c541ce92b5 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:00.626155 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a14e2e7-b21b-4a97-9f3b-f0b5915dc57a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:00.633008 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a14e2e7-b21b-4a97-9f3b-f0b5915dc57a admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:00.633008 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a14e2e7-b21b-4a97-9f3b-f0b5915dc57a admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:00.633008 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a14e2e7-b21b-4a97-9f3b-f0b5915dc57a admin admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:00.633008 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a14e2e7-b21b-4a97-9f3b-f0b5915dc57a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:20:00.633008 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a14e2e7-b21b-4a97-9f3b-f0b5915dc57a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:00.633008 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a14e2e7-b21b-4a97-9f3b-f0b5915dc57a admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:00.651099 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 984/1955] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Aug 25 03:20:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:20:00.656134 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62e69b8-f527-4312-a8f6-da0f855d1ada None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:00.672043 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62e69b8-f527-4312-a8f6-da0f855d1ada None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:00.677528 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0e66d2-cc45-47a5-8bd2-1767da0df647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:00.678459 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0e66d2-cc45-47a5-8bd2-1767da0df647 admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:00.678980 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0e66d2-cc45-47a5-8bd2-1767da0df647 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:00.680834 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0e66d2-cc45-47a5-8bd2-1767da0df647 admin admin] PATH_INFO: `/v3/projects` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:00.681489 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0e66d2-cc45-47a5-8bd2-1767da0df647 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:00.683464 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0e66d2-cc45-47a5-8bd2-1767da0df647 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:00.803599 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 972/1956] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:20:00 2022] POST /identity/v3/projects => generated 373 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:20:00.806419 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a14e2e7-b21b-4a97-9f3b-f0b5915dc57a admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:00.817158 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a14e2e7-b21b-4a97-9f3b-f0b5915dc57a admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:00.917826 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7382e2-5ac6-4de5-8dc2-25290375cdfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:00.923607 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7382e2-5ac6-4de5-8dc2-25290375cdfc admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:00.923607 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7382e2-5ac6-4de5-8dc2-25290375cdfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:00.923607 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7382e2-5ac6-4de5-8dc2-25290375cdfc admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:00.923607 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7382e2-5ac6-4de5-8dc2-25290375cdfc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:00.923607 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7382e2-5ac6-4de5-8dc2-25290375cdfc admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:01.042156 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 985/1957] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:20:00 2022] POST /identity/v3/users => generated 287 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:20:01.046354 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0e66d2-cc45-47a5-8bd2-1767da0df647 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:01.056997 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0e66d2-cc45-47a5-8bd2-1767da0df647 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:01.071515 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf55883-72cf-40e2-b932-185375dca361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:01.074410 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf55883-72cf-40e2-b932-185375dca361 admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:01.074410 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf55883-72cf-40e2-b932-185375dca361 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:01.074410 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf55883-72cf-40e2-b932-185375dca361 admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:01.074410 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf55883-72cf-40e2-b932-185375dca361 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:01.081202 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf55883-72cf-40e2-b932-185375dca361 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:01.095054 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 973/1958] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:20:01 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:20:01.098952 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7382e2-5ac6-4de5-8dc2-25290375cdfc admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:01.112948 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7382e2-5ac6-4de5-8dc2-25290375cdfc admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:01.118890 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5216940-57ba-4f69-b597-94528eabf445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:01.119513 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5216940-57ba-4f69-b597-94528eabf445 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:01.119802 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5216940-57ba-4f69-b597-94528eabf445 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:01.120082 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5216940-57ba-4f69-b597-94528eabf445 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:01.121268 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5216940-57ba-4f69-b597-94528eabf445 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:01.123094 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5216940-57ba-4f69-b597-94528eabf445 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:01.139681 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 986/1959] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:20:01 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:20:01.145603 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf55883-72cf-40e2-b932-185375dca361 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:01.176023 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf55883-72cf-40e2-b932-185375dca361 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:01.176410 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb194474-a7c1-4378-bf86-0668422108c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:01.177115 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb194474-a7c1-4378-bf86-0668422108c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:01.177415 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb194474-a7c1-4378-bf86-0668422108c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:01.177782 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb194474-a7c1-4378-bf86-0668422108c0 admin admin] PATH_INFO: `/v3/projects/882db843d4644ac29365868d1deb9a27/users/2552825edca0418392f877ea9488e36f/roles/3255f77192524a16bc405173106871fc` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:01.196362 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb194474-a7c1-4378-bf86-0668422108c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=882db843d4644ac29365868d1deb9a27, user_id=2552825edca0418392f877ea9488e36f, role_id=3255f77192524a16bc405173106871fc)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:01.200477 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb194474-a7c1-4378-bf86-0668422108c0 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:01.298119 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 974/1960] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:20:01 2022] PUT /identity/v3/projects/882db843d4644ac29365868d1deb9a27/users/2552825edca0418392f877ea9488e36f/roles/3255f77192524a16bc405173106871fc => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:20:01.303305 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecccca9-ca97-44de-b557-53d864deefb0 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:01.303598 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecccca9-ca97-44de-b557-53d864deefb0 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:01.303908 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecccca9-ca97-44de-b557-53d864deefb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:01.325511 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-4ecccca9-ca97-44de-b557-53d864deefb0 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:20:01.325511 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-4ecccca9-ca97-44de-b557-53d864deefb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:01.388544 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 987/1961] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:20:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:20:01.399554 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb194474-a7c1-4378-bf86-0668422108c0 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:01.401237 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-cb194474-a7c1-4378-bf86-0668422108c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:01.446272 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:20:01.457220 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb194474-a7c1-4378-bf86-0668422108c0 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:01.460934 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55300203-7fb5-40ad-92b5-eecbfb2b5f0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:01.461552 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55300203-7fb5-40ad-92b5-eecbfb2b5f0d service neutron] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:01.461831 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55300203-7fb5-40ad-92b5-eecbfb2b5f0d service neutron] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:01.462137 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55300203-7fb5-40ad-92b5-eecbfb2b5f0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:01.471884 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55300203-7fb5-40ad-92b5-eecbfb2b5f0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:01.473373 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55300203-7fb5-40ad-92b5-eecbfb2b5f0d service neutron] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:01.483867 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 975/1962] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:20:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:20:02.859500 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdd5e85-3242-4e04-af6e-c92cd7918332 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:02.860012 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdd5e85-3242-4e04-af6e-c92cd7918332 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:02.860218 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdd5e85-3242-4e04-af6e-c92cd7918332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:02.886195 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:20:02.889810 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-dfdd5e85-3242-4e04-af6e-c92cd7918332 None None] MFA Rules not processed for user `2552825edca0418392f877ea9488e36f`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:20:02.891345 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-dfdd5e85-3242-4e04-af6e-c92cd7918332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:02.963227 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 988/1963] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:20:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:20:02.974852 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55300203-7fb5-40ad-92b5-eecbfb2b5f0d service neutron] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:02.976404 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-55300203-7fb5-40ad-92b5-eecbfb2b5f0d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:03.032186 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55300203-7fb5-40ad-92b5-eecbfb2b5f0d service neutron] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:03.036340 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd3902a-7493-446f-9153-d6d69962bdf4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60193ba9e1c14df3bc9eebcc6f28f2d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:03.036944 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd3902a-7493-446f-9153-d6d69962bdf4 service cinder] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:03.037228 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd3902a-7493-446f-9153-d6d69962bdf4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:03.037506 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd3902a-7493-446f-9153-d6d69962bdf4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:03.045920 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd3902a-7493-446f-9153-d6d69962bdf4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:03.047379 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd3902a-7493-446f-9153-d6d69962bdf4 service cinder] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:03.056582 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 976/1964] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:20:02 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:20:07.914172 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df71a99d-e2f4-47c5-98ca-888002660d73 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:07.914793 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df71a99d-e2f4-47c5-98ca-888002660d73 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:07.914793 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df71a99d-e2f4-47c5-98ca-888002660d73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:07.938707 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-df71a99d-e2f4-47c5-98ca-888002660d73 None None] MFA Rules not processed for user `2552825edca0418392f877ea9488e36f`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:20:07.940019 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-df71a99d-e2f4-47c5-98ca-888002660d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:07.947995 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 989/1965] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:20:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:20:08.699942 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd3902a-7493-446f-9153-d6d69962bdf4 service cinder] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:08.701468 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-9fd3902a-7493-446f-9153-d6d69962bdf4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:08.725742 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd3902a-7493-446f-9153-d6d69962bdf4 service cinder] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:08.737007 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fb3739-9201-4fd8-b783-d98848d87e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:08.737881 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fb3739-9201-4fd8-b783-d98848d87e93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:08.738405 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fb3739-9201-4fd8-b783-d98848d87e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:08.738866 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fb3739-9201-4fd8-b783-d98848d87e93 admin admin] PATH_INFO: `/v3/users/fca51c4680ee4580a94e27bbd292318b` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:08.740371 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fb3739-9201-4fd8-b783-d98848d87e93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fca51c4680ee4580a94e27bbd292318b)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:08.742885 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fb3739-9201-4fd8-b783-d98848d87e93 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:09.032274 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-c2fb3739-9201-4fd8-b783-d98848d87e93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fca51c4680ee4580a94e27bbd292318b'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:09.161046 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-c2fb3739-9201-4fd8-b783-d98848d87e93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fca51c4680ee4580a94e27bbd292318b'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:09.161679 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-c2fb3739-9201-4fd8-b783-d98848d87e93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fca51c4680ee4580a94e27bbd292318b'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:09.163187 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-c2fb3739-9201-4fd8-b783-d98848d87e93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fca51c4680ee4580a94e27bbd292318b'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:09.182801 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-c2fb3739-9201-4fd8-b783-d98848d87e93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fca51c4680ee4580a94e27bbd292318b'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:09.196533 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 977/1966] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:20:08 2022] DELETE /identity/v3/users/fca51c4680ee4580a94e27bbd292318b => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:20:09.201581 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649c1d64-df84-474c-95b4-82704a6ee1d9 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:09.202436 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649c1d64-df84-474c-95b4-82704a6ee1d9 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:09.202844 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649c1d64-df84-474c-95b4-82704a6ee1d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:09.222091 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-649c1d64-df84-474c-95b4-82704a6ee1d9 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:20:09.223363 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-649c1d64-df84-474c-95b4-82704a6ee1d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:09.304334 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 990/1967] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:20:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:20:09.315113 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fb3739-9201-4fd8-b783-d98848d87e93 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:09.316918 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-c2fb3739-9201-4fd8-b783-d98848d87e93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:09.377568 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fb3739-9201-4fd8-b783-d98848d87e93 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:09.381415 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca53ec29-53df-4be8-b5fc-075de61fc7ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:09.382036 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca53ec29-53df-4be8-b5fc-075de61fc7ff service neutron] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:09.382346 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca53ec29-53df-4be8-b5fc-075de61fc7ff service neutron] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:09.382625 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca53ec29-53df-4be8-b5fc-075de61fc7ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:09.392175 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca53ec29-53df-4be8-b5fc-075de61fc7ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:09.393836 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca53ec29-53df-4be8-b5fc-075de61fc7ff service neutron] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:09.403156 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 978/1968] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:20:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:20:09.473526 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649c1d64-df84-474c-95b4-82704a6ee1d9 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:09.473526 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-649c1d64-df84-474c-95b4-82704a6ee1d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:09.483684 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649c1d64-df84-474c-95b4-82704a6ee1d9 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:09.488450 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d51c21-1c8b-47a9-b8cd-b95329e4fa5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:09.489085 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d51c21-1c8b-47a9-b8cd-b95329e4fa5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:09.489369 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d51c21-1c8b-47a9-b8cd-b95329e4fa5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:09.489646 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d51c21-1c8b-47a9-b8cd-b95329e4fa5b admin admin] PATH_INFO: `/v3/users/5fb7ef699a044b8d956a0c02a53777a0` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:09.490878 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d51c21-1c8b-47a9-b8cd-b95329e4fa5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fb7ef699a044b8d956a0c02a53777a0)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:09.492799 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d51c21-1c8b-47a9-b8cd-b95329e4fa5b admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:09.729683 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-d6d51c21-1c8b-47a9-b8cd-b95329e4fa5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fb7ef699a044b8d956a0c02a53777a0'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:09.824851 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-d6d51c21-1c8b-47a9-b8cd-b95329e4fa5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fb7ef699a044b8d956a0c02a53777a0'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:09.825587 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-d6d51c21-1c8b-47a9-b8cd-b95329e4fa5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fb7ef699a044b8d956a0c02a53777a0'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:09.836604 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-d6d51c21-1c8b-47a9-b8cd-b95329e4fa5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fb7ef699a044b8d956a0c02a53777a0'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:09.851706 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-d6d51c21-1c8b-47a9-b8cd-b95329e4fa5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fb7ef699a044b8d956a0c02a53777a0'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:09.861181 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 991/1969] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:20:09 2022] DELETE /identity/v3/users/5fb7ef699a044b8d956a0c02a53777a0 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:20:09.863264 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c46b151-7318-4143-bcc7-8fc0a61896b2 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:09.863731 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c46b151-7318-4143-bcc7-8fc0a61896b2 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:09.863835 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c46b151-7318-4143-bcc7-8fc0a61896b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:09.871893 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:20:09.883253 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-5c46b151-7318-4143-bcc7-8fc0a61896b2 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:20:09.886218 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-5c46b151-7318-4143-bcc7-8fc0a61896b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:09.911892 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d51c21-1c8b-47a9-b8cd-b95329e4fa5b admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:09.913385 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-d6d51c21-1c8b-47a9-b8cd-b95329e4fa5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:09.927879 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:20:09.953916 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 979/1970] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:20:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:20:09.964256 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c46b151-7318-4143-bcc7-8fc0a61896b2 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:09.966470 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-5c46b151-7318-4143-bcc7-8fc0a61896b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:09.988149 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d51c21-1c8b-47a9-b8cd-b95329e4fa5b admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:09.993224 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491c33a3-d3c9-4a1d-a35a-2cabd1a097c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:09.993983 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491c33a3-d3c9-4a1d-a35a-2cabd1a097c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:09.994529 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491c33a3-d3c9-4a1d-a35a-2cabd1a097c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:09.995256 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491c33a3-d3c9-4a1d-a35a-2cabd1a097c4 admin admin] PATH_INFO: `/v3/projects/130bfeb2c52b4caaaac9b01c565a3afb` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:09.997353 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491c33a3-d3c9-4a1d-a35a-2cabd1a097c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=130bfeb2c52b4caaaac9b01c565a3afb)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:09.999348 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491c33a3-d3c9-4a1d-a35a-2cabd1a097c4 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:10.027616 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c46b151-7318-4143-bcc7-8fc0a61896b2 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:10.031383 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7067641f-5eaa-4058-8d8f-6758bb0cd877 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:10.032000 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7067641f-5eaa-4058-8d8f-6758bb0cd877 service neutron] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:10.032289 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7067641f-5eaa-4058-8d8f-6758bb0cd877 service neutron] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:10.032568 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7067641f-5eaa-4058-8d8f-6758bb0cd877 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:10.042553 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7067641f-5eaa-4058-8d8f-6758bb0cd877 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:10.044043 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7067641f-5eaa-4058-8d8f-6758bb0cd877 service neutron] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:10.053484 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 980/1971] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:20:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 25 03:20:10.142926 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-491c33a3-d3c9-4a1d-a35a-2cabd1a097c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '130bfeb2c52b4caaaac9b01c565a3afb', 'request_id': 'req-491c33a3-d3c9-4a1d-a35a-2cabd1a097c4'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:10.275182 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-491c33a3-d3c9-4a1d-a35a-2cabd1a097c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '130bfeb2c52b4caaaac9b01c565a3afb', 'request_id': 'req-491c33a3-d3c9-4a1d-a35a-2cabd1a097c4'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:10.282748 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-491c33a3-d3c9-4a1d-a35a-2cabd1a097c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '130bfeb2c52b4caaaac9b01c565a3afb', 'request_id': 'req-491c33a3-d3c9-4a1d-a35a-2cabd1a097c4'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:10.308110 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-491c33a3-d3c9-4a1d-a35a-2cabd1a097c4 admin admin] The token cache is being invalidate because project 130bfeb2c52b4caaaac9b01c565a3afb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:20:10.308755 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-491c33a3-d3c9-4a1d-a35a-2cabd1a097c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:10.310511 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-491c33a3-d3c9-4a1d-a35a-2cabd1a097c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:10.313270 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 992/1972] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:20:09 2022] DELETE /identity/v3/projects/130bfeb2c52b4caaaac9b01c565a3afb => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:20:10.318462 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7067641f-5eaa-4058-8d8f-6758bb0cd877 service neutron] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:10.328927 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7067641f-5eaa-4058-8d8f-6758bb0cd877 service neutron] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:10.424125 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5233ac-23d5-4458-926a-7725777e796b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:10.424782 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5233ac-23d5-4458-926a-7725777e796b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:10.425064 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5233ac-23d5-4458-926a-7725777e796b admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:10.425339 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5233ac-23d5-4458-926a-7725777e796b admin admin] PATH_INFO: `/v3/users/177e443e234a4f8d834cee618dec014c` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:10.426922 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5233ac-23d5-4458-926a-7725777e796b admin admin] RBAC: Authorizing `identity:delete_user(user_id=177e443e234a4f8d834cee618dec014c)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:10.430312 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5233ac-23d5-4458-926a-7725777e796b admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:10.573494 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491c33a3-d3c9-4a1d-a35a-2cabd1a097c4 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:10.575562 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-491c33a3-d3c9-4a1d-a35a-2cabd1a097c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:10.588883 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491c33a3-d3c9-4a1d-a35a-2cabd1a097c4 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:10.593541 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e351bb2a-0e22-4ed6-991b-449456e8135a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:10.594185 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e351bb2a-0e22-4ed6-991b-449456e8135a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:10.594498 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e351bb2a-0e22-4ed6-991b-449456e8135a admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:10.594776 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e351bb2a-0e22-4ed6-991b-449456e8135a admin admin] PATH_INFO: `/v3/projects/da4d7a1bdb68408182e7982ce684907c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:10.596414 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e351bb2a-0e22-4ed6-991b-449456e8135a admin admin] RBAC: Authorizing `identity:delete_project(project_id=da4d7a1bdb68408182e7982ce684907c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:10.598387 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e351bb2a-0e22-4ed6-991b-449456e8135a admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:10.665577 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-6e5233ac-23d5-4458-926a-7725777e796b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '177e443e234a4f8d834cee618dec014c'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:10.668162 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:20:10.726678 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-e351bb2a-0e22-4ed6-991b-449456e8135a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da4d7a1bdb68408182e7982ce684907c', 'request_id': 'req-e351bb2a-0e22-4ed6-991b-449456e8135a'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:10.737826 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-6e5233ac-23d5-4458-926a-7725777e796b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '177e443e234a4f8d834cee618dec014c'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:10.738537 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-6e5233ac-23d5-4458-926a-7725777e796b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '177e443e234a4f8d834cee618dec014c'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:10.739146 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-6e5233ac-23d5-4458-926a-7725777e796b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '177e443e234a4f8d834cee618dec014c'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:10.751766 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-6e5233ac-23d5-4458-926a-7725777e796b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '177e443e234a4f8d834cee618dec014c'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:10.765455 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 981/1973] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:20:10 2022] DELETE /identity/v3/users/177e443e234a4f8d834cee618dec014c => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:20:10.817693 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-e351bb2a-0e22-4ed6-991b-449456e8135a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da4d7a1bdb68408182e7982ce684907c', 'request_id': 'req-e351bb2a-0e22-4ed6-991b-449456e8135a'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:10.825289 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-e351bb2a-0e22-4ed6-991b-449456e8135a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da4d7a1bdb68408182e7982ce684907c', 'request_id': 'req-e351bb2a-0e22-4ed6-991b-449456e8135a'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:10.842531 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-e351bb2a-0e22-4ed6-991b-449456e8135a admin admin] The token cache is being invalidate because project da4d7a1bdb68408182e7982ce684907c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:20:10.843575 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-e351bb2a-0e22-4ed6-991b-449456e8135a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:10.844545 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-e351bb2a-0e22-4ed6-991b-449456e8135a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:10.847583 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 993/1974] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:20:10 2022] DELETE /identity/v3/projects/da4d7a1bdb68408182e7982ce684907c => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:20:10.851609 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5233ac-23d5-4458-926a-7725777e796b admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:10.853260 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-6e5233ac-23d5-4458-926a-7725777e796b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:10.930099 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5233ac-23d5-4458-926a-7725777e796b admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:10.945790 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd7bc47-1aea-4557-87f0-19e44fdb412a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:10.947112 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd7bc47-1aea-4557-87f0-19e44fdb412a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:10.947405 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd7bc47-1aea-4557-87f0-19e44fdb412a admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:10.948219 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd7bc47-1aea-4557-87f0-19e44fdb412a admin admin] PATH_INFO: `/v3/users/6d673ef3236643378c2e5101c354d59b` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:10.951803 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd7bc47-1aea-4557-87f0-19e44fdb412a admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d673ef3236643378c2e5101c354d59b)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:10.955031 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd7bc47-1aea-4557-87f0-19e44fdb412a admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:11.190788 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e351bb2a-0e22-4ed6-991b-449456e8135a admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:11.192728 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-e351bb2a-0e22-4ed6-991b-449456e8135a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:11.202821 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-2fd7bc47-1aea-4557-87f0-19e44fdb412a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d673ef3236643378c2e5101c354d59b'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:11.228297 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:20:11.265113 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e351bb2a-0e22-4ed6-991b-449456e8135a admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:11.270226 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a2af68-546e-47ac-a7b3-dd9ea0775116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:11.270885 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a2af68-546e-47ac-a7b3-dd9ea0775116 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:11.271162 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a2af68-546e-47ac-a7b3-dd9ea0775116 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:11.271436 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a2af68-546e-47ac-a7b3-dd9ea0775116 admin admin] PATH_INFO: `/v3/projects/612b0f3a42864fd58919a2d49396d334` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:11.272669 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a2af68-546e-47ac-a7b3-dd9ea0775116 admin admin] RBAC: Authorizing `identity:delete_project(project_id=612b0f3a42864fd58919a2d49396d334)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:11.274667 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a2af68-546e-47ac-a7b3-dd9ea0775116 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:11.285980 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-2fd7bc47-1aea-4557-87f0-19e44fdb412a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d673ef3236643378c2e5101c354d59b'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:11.287311 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-2fd7bc47-1aea-4557-87f0-19e44fdb412a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d673ef3236643378c2e5101c354d59b'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:11.287951 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-2fd7bc47-1aea-4557-87f0-19e44fdb412a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d673ef3236643378c2e5101c354d59b'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:11.304738 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-2fd7bc47-1aea-4557-87f0-19e44fdb412a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d673ef3236643378c2e5101c354d59b'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:11.317373 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 982/1975] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:20:10 2022] DELETE /identity/v3/users/6d673ef3236643378c2e5101c354d59b => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:20:11.403134 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-83a2af68-546e-47ac-a7b3-dd9ea0775116 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '612b0f3a42864fd58919a2d49396d334', 'request_id': 'req-83a2af68-546e-47ac-a7b3-dd9ea0775116'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:11.469124 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-83a2af68-546e-47ac-a7b3-dd9ea0775116 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '612b0f3a42864fd58919a2d49396d334', 'request_id': 'req-83a2af68-546e-47ac-a7b3-dd9ea0775116'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:11.479334 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-83a2af68-546e-47ac-a7b3-dd9ea0775116 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '612b0f3a42864fd58919a2d49396d334', 'request_id': 'req-83a2af68-546e-47ac-a7b3-dd9ea0775116'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:11.492905 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-83a2af68-546e-47ac-a7b3-dd9ea0775116 admin admin] The token cache is being invalidate because project 612b0f3a42864fd58919a2d49396d334 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:20:11.493268 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-83a2af68-546e-47ac-a7b3-dd9ea0775116 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:11.494102 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-83a2af68-546e-47ac-a7b3-dd9ea0775116 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:11.496043 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 994/1976] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:20:11 2022] DELETE /identity/v3/projects/612b0f3a42864fd58919a2d49396d334 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:20:11.502499 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738ffe53-c79e-4df1-b3ea-a0dc85f7740c None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:11.502906 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738ffe53-c79e-4df1-b3ea-a0dc85f7740c None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:11.503299 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738ffe53-c79e-4df1-b3ea-a0dc85f7740c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:11.534160 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-738ffe53-c79e-4df1-b3ea-a0dc85f7740c None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:20:11.535880 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-738ffe53-c79e-4df1-b3ea-a0dc85f7740c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:11.627652 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:20:11.682188 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36040f5f-4ca7-42de-8d25-8cbac29daa07 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:11.682746 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36040f5f-4ca7-42de-8d25-8cbac29daa07 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:11.683108 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36040f5f-4ca7-42de-8d25-8cbac29daa07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:11.689615 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 983/1977] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:20:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:20:11.724090 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-36040f5f-4ca7-42de-8d25-8cbac29daa07 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:20:11.725498 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-36040f5f-4ca7-42de-8d25-8cbac29daa07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:11.752486 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 995/1978] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:20:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:20:11.758008 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738ffe53-c79e-4df1-b3ea-a0dc85f7740c None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:11.770895 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738ffe53-c79e-4df1-b3ea-a0dc85f7740c None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:11.777177 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d29d095-95ef-4781-a335-48b55083f95a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:11.777773 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d29d095-95ef-4781-a335-48b55083f95a admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:11.778128 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d29d095-95ef-4781-a335-48b55083f95a admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:11.778555 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d29d095-95ef-4781-a335-48b55083f95a admin admin] PATH_INFO: `/v3/domains` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:11.779393 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d29d095-95ef-4781-a335-48b55083f95a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:20:11.779771 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d29d095-95ef-4781-a335-48b55083f95a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:11.781630 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d29d095-95ef-4781-a335-48b55083f95a admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:11.810576 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 984/1979] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Aug 25 03:20:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:20:11.811666 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36040f5f-4ca7-42de-8d25-8cbac29daa07 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:11.826154 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36040f5f-4ca7-42de-8d25-8cbac29daa07 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:11.835643 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fabd65d-8f63-49e4-93cb-e7b1ce9f451b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:11.835643 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fabd65d-8f63-49e4-93cb-e7b1ce9f451b admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:11.835643 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fabd65d-8f63-49e4-93cb-e7b1ce9f451b admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:11.835643 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fabd65d-8f63-49e4-93cb-e7b1ce9f451b admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:11.835936 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fabd65d-8f63-49e4-93cb-e7b1ce9f451b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:11.835936 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fabd65d-8f63-49e4-93cb-e7b1ce9f451b admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:11.838357 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d29d095-95ef-4781-a335-48b55083f95a admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:11.839811 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-9d29d095-95ef-4781-a335-48b55083f95a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:11.870271 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d29d095-95ef-4781-a335-48b55083f95a admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:11.879813 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c61f339-8749-4914-b7c7-671608bd00f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:11.880515 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c61f339-8749-4914-b7c7-671608bd00f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:11.880806 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c61f339-8749-4914-b7c7-671608bd00f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:11.881077 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c61f339-8749-4914-b7c7-671608bd00f1 admin admin] PATH_INFO: `/v3/projects/ebda46e793bf49599fc83fe4fb7a337d` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:11.882352 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c61f339-8749-4914-b7c7-671608bd00f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebda46e793bf49599fc83fe4fb7a337d)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:11.884342 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c61f339-8749-4914-b7c7-671608bd00f1 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:12.012977 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 996/1980] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:20:11 2022] POST /identity/v3/projects => generated 383 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:20:12.016573 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fabd65d-8f63-49e4-93cb-e7b1ce9f451b admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:12.027365 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-7c61f339-8749-4914-b7c7-671608bd00f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebda46e793bf49599fc83fe4fb7a337d', 'request_id': 'req-7c61f339-8749-4914-b7c7-671608bd00f1'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:12.083456 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fabd65d-8f63-49e4-93cb-e7b1ce9f451b admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:12.088626 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b30d0d-63ed-4443-bd90-7b46236faa35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:12.089251 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b30d0d-63ed-4443-bd90-7b46236faa35 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:12.089535 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b30d0d-63ed-4443-bd90-7b46236faa35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:12.089833 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b30d0d-63ed-4443-bd90-7b46236faa35 admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:12.090494 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b30d0d-63ed-4443-bd90-7b46236faa35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:12.092499 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b30d0d-63ed-4443-bd90-7b46236faa35 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:12.194610 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-7c61f339-8749-4914-b7c7-671608bd00f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebda46e793bf49599fc83fe4fb7a337d', 'request_id': 'req-7c61f339-8749-4914-b7c7-671608bd00f1'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:12.200695 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-7c61f339-8749-4914-b7c7-671608bd00f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebda46e793bf49599fc83fe4fb7a337d', 'request_id': 'req-7c61f339-8749-4914-b7c7-671608bd00f1'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:12.212000 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-7c61f339-8749-4914-b7c7-671608bd00f1 admin admin] The token cache is being invalidate because project ebda46e793bf49599fc83fe4fb7a337d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81995) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:20:12.212309 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-7c61f339-8749-4914-b7c7-671608bd00f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:12.212934 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-7c61f339-8749-4914-b7c7-671608bd00f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:12.214492 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 985/1981] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:20:11 2022] DELETE /identity/v3/projects/ebda46e793bf49599fc83fe4fb7a337d => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:20:12.216043 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c61f339-8749-4914-b7c7-671608bd00f1 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:12.216525 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-7c61f339-8749-4914-b7c7-671608bd00f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:12.236721 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 997/1982] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:20:12 2022] POST /identity/v3/users => generated 292 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:20:12.238712 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd95435-80b7-443a-a448-562e93643235 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:12.239000 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd95435-80b7-443a-a448-562e93643235 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:12.239284 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd95435-80b7-443a-a448-562e93643235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:12.262983 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-2bd95435-80b7-443a-a448-562e93643235 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:20:12.262983 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-2bd95435-80b7-443a-a448-562e93643235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:12.270082 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 998/1983] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:20:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:20:12.278818 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd95435-80b7-443a-a448-562e93643235 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:12.278818 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-2bd95435-80b7-443a-a448-562e93643235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:12.285309 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd95435-80b7-443a-a448-562e93643235 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:12.289965 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1a8c63-6989-4399-a813-6db2198d15d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:12.290599 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1a8c63-6989-4399-a813-6db2198d15d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:12.290879 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1a8c63-6989-4399-a813-6db2198d15d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:12.291155 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1a8c63-6989-4399-a813-6db2198d15d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:12.291841 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1a8c63-6989-4399-a813-6db2198d15d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:12.293383 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1a8c63-6989-4399-a813-6db2198d15d8 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:12.294951 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:20:12.302606 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c61f339-8749-4914-b7c7-671608bd00f1 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:12.308310 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 999/1984] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:20:12 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:20:12.308310 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c4ce14-9571-46fe-9b5a-07b96b4b8197 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:12.308310 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c4ce14-9571-46fe-9b5a-07b96b4b8197 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:12.308310 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c4ce14-9571-46fe-9b5a-07b96b4b8197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:12.310533 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c9bbd9-0895-4292-a9b1-981b5ed03b07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:12.311179 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c9bbd9-0895-4292-a9b1-981b5ed03b07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:12.311463 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c9bbd9-0895-4292-a9b1-981b5ed03b07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:12.311743 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c9bbd9-0895-4292-a9b1-981b5ed03b07 admin admin] PATH_INFO: `/v3/users/e501606859504c89a896d6b19afbb21a` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:12.312904 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c9bbd9-0895-4292-a9b1-981b5ed03b07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e501606859504c89a896d6b19afbb21a)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:12.315007 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c9bbd9-0895-4292-a9b1-981b5ed03b07 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:12.325743 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-52c4ce14-9571-46fe-9b5a-07b96b4b8197 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:20:12.327001 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-52c4ce14-9571-46fe-9b5a-07b96b4b8197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:12.335142 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1000/1985] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:20:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:20:12.337819 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c4ce14-9571-46fe-9b5a-07b96b4b8197 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:12.348956 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c4ce14-9571-46fe-9b5a-07b96b4b8197 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:12.353559 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d92cfb-663c-469c-ac37-a3f289a7b209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:12.354179 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d92cfb-663c-469c-ac37-a3f289a7b209 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:12.354494 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d92cfb-663c-469c-ac37-a3f289a7b209 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:12.354766 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d92cfb-663c-469c-ac37-a3f289a7b209 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:12.355402 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d92cfb-663c-469c-ac37-a3f289a7b209 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:12.356978 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d92cfb-663c-469c-ac37-a3f289a7b209 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:12.367595 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1001/1986] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:20:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:20:12.368369 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d92cfb-663c-469c-ac37-a3f289a7b209 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:12.377263 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d92cfb-663c-469c-ac37-a3f289a7b209 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:12.383345 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31268d1f-39cd-43f9-8dfe-efb46e83e743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:12.383345 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31268d1f-39cd-43f9-8dfe-efb46e83e743 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:12.383345 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31268d1f-39cd-43f9-8dfe-efb46e83e743 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:12.383345 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31268d1f-39cd-43f9-8dfe-efb46e83e743 admin admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:12.383710 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31268d1f-39cd-43f9-8dfe-efb46e83e743 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:20:12.383891 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31268d1f-39cd-43f9-8dfe-efb46e83e743 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:12.385411 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31268d1f-39cd-43f9-8dfe-efb46e83e743 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:12.397013 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1002/1987] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Aug 25 03:20:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:20:12.397752 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31268d1f-39cd-43f9-8dfe-efb46e83e743 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:12.406623 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31268d1f-39cd-43f9-8dfe-efb46e83e743 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:12.412090 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d235a6-7c34-4152-bcad-4a059e780a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:12.412481 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d235a6-7c34-4152-bcad-4a059e780a8e admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:12.412765 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d235a6-7c34-4152-bcad-4a059e780a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:12.413043 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d235a6-7c34-4152-bcad-4a059e780a8e admin admin] PATH_INFO: `/v3/projects/1cf782e16e5d4d6bbcf56bec07a1c506/users/5adc372e2fdc45ec97c02ca7f90f1ffc/roles/3255f77192524a16bc405173106871fc` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:12.427644 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d235a6-7c34-4152-bcad-4a059e780a8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cf782e16e5d4d6bbcf56bec07a1c506, user_id=5adc372e2fdc45ec97c02ca7f90f1ffc, role_id=3255f77192524a16bc405173106871fc)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:12.430304 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d235a6-7c34-4152-bcad-4a059e780a8e admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:12.522793 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-03c9bbd9-0895-4292-a9b1-981b5ed03b07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e501606859504c89a896d6b19afbb21a'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:12.543741 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1003/1988] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:20:12 2022] PUT /identity/v3/projects/1cf782e16e5d4d6bbcf56bec07a1c506/users/5adc372e2fdc45ec97c02ca7f90f1ffc/roles/3255f77192524a16bc405173106871fc => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:20:12.544614 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d235a6-7c34-4152-bcad-4a059e780a8e admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:12.612101 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d235a6-7c34-4152-bcad-4a059e780a8e admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:12.616760 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543b925b-87a1-404a-bcbc-568cfafa1e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:12.616939 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-03c9bbd9-0895-4292-a9b1-981b5ed03b07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e501606859504c89a896d6b19afbb21a'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:12.617370 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543b925b-87a1-404a-bcbc-568cfafa1e70 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:12.617481 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-03c9bbd9-0895-4292-a9b1-981b5ed03b07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e501606859504c89a896d6b19afbb21a'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:12.617719 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543b925b-87a1-404a-bcbc-568cfafa1e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:12.617934 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543b925b-87a1-404a-bcbc-568cfafa1e70 admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:12.618062 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-03c9bbd9-0895-4292-a9b1-981b5ed03b07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e501606859504c89a896d6b19afbb21a'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:12.618596 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543b925b-87a1-404a-bcbc-568cfafa1e70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:12.620496 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543b925b-87a1-404a-bcbc-568cfafa1e70 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:12.623095 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:20:12.635342 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-03c9bbd9-0895-4292-a9b1-981b5ed03b07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e501606859504c89a896d6b19afbb21a'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:12.645023 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 986/1989] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:20:12 2022] DELETE /identity/v3/users/e501606859504c89a896d6b19afbb21a => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:20:12.646560 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa82a79-f535-438d-973a-6dfcb29020d2 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:12.646857 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa82a79-f535-438d-973a-6dfcb29020d2 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:12.647136 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa82a79-f535-438d-973a-6dfcb29020d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:12.675059 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-2fa82a79-f535-438d-973a-6dfcb29020d2 None None] MFA Rules not processed for user `5adc372e2fdc45ec97c02ca7f90f1ffc`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:20:12.676334 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-2fa82a79-f535-438d-973a-6dfcb29020d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:12.715884 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1004/1990] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:20:12 2022] POST /identity/v3/projects => generated 381 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:20:12.717179 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144028ee-ffd7-4406-b63f-6addda82f74c None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:12.717578 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144028ee-ffd7-4406-b63f-6addda82f74c None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:12.717731 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144028ee-ffd7-4406-b63f-6addda82f74c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:12.737086 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-144028ee-ffd7-4406-b63f-6addda82f74c None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:20:12.738273 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-144028ee-ffd7-4406-b63f-6addda82f74c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:12.741454 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 987/1991] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:20:12 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:20:12.745210 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa82a79-f535-438d-973a-6dfcb29020d2 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:12.745210 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-2fa82a79-f535-438d-973a-6dfcb29020d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:12.810088 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1005/1992] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:20:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:20:12.810592 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144028ee-ffd7-4406-b63f-6addda82f74c None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:12.823431 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-144028ee-ffd7-4406-b63f-6addda82f74c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:12.837823 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144028ee-ffd7-4406-b63f-6addda82f74c None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:12.845931 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c93b111-f7e3-4976-bb51-f1da9a90f3e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:12.846611 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c93b111-f7e3-4976-bb51-f1da9a90f3e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:12.846916 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c93b111-f7e3-4976-bb51-f1da9a90f3e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:12.847336 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c93b111-f7e3-4976-bb51-f1da9a90f3e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:12.853450 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c93b111-f7e3-4976-bb51-f1da9a90f3e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:12.853450 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c93b111-f7e3-4976-bb51-f1da9a90f3e5 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:12.857959 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa82a79-f535-438d-973a-6dfcb29020d2 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:12.880239 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0401fc0b-0e23-4bd9-a52c-7e3dd31c5610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:12.880816 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0401fc0b-0e23-4bd9-a52c-7e3dd31c5610 admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:12.881092 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0401fc0b-0e23-4bd9-a52c-7e3dd31c5610 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:12.881365 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0401fc0b-0e23-4bd9-a52c-7e3dd31c5610 admin admin] PATH_INFO: `/v3/users` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:12.882132 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0401fc0b-0e23-4bd9-a52c-7e3dd31c5610 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:12.884129 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0401fc0b-0e23-4bd9-a52c-7e3dd31c5610 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:12.968189 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 988/1993] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:20:12 2022] POST /identity/v3/users => generated 291 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:20:12.969236 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1006/1994] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:20:12 2022] POST /identity/v3/projects => generated 385 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:20:12.969466 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0401fc0b-0e23-4bd9-a52c-7e3dd31c5610 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:12.972677 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c93b111-f7e3-4976-bb51-f1da9a90f3e5 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:12.976870 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-0401fc0b-0e23-4bd9-a52c-7e3dd31c5610 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:12.979036 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:20:13.056483 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c93b111-f7e3-4976-bb51-f1da9a90f3e5 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:13.061937 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708fae33-20e1-4cf0-9069-67b6fd26fb48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:13.062595 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708fae33-20e1-4cf0-9069-67b6fd26fb48 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:13.062877 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708fae33-20e1-4cf0-9069-67b6fd26fb48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:13.063158 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708fae33-20e1-4cf0-9069-67b6fd26fb48 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:13.063802 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708fae33-20e1-4cf0-9069-67b6fd26fb48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:13.066089 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708fae33-20e1-4cf0-9069-67b6fd26fb48 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:13.066626 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0401fc0b-0e23-4bd9-a52c-7e3dd31c5610 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:13.080323 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1007/1995] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:20:12 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:20:13.080323 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708fae33-20e1-4cf0-9069-67b6fd26fb48 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:13.080551 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:20:13.088224 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708fae33-20e1-4cf0-9069-67b6fd26fb48 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:13.095199 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb7ae75-68e2-4f7e-a5e1-9a19da013733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:13.095805 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb7ae75-68e2-4f7e-a5e1-9a19da013733 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:13.096082 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb7ae75-68e2-4f7e-a5e1-9a19da013733 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:13.096359 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb7ae75-68e2-4f7e-a5e1-9a19da013733 admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:13.096979 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb7ae75-68e2-4f7e-a5e1-9a19da013733 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:13.110957 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb7ae75-68e2-4f7e-a5e1-9a19da013733 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:13.124585 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96b2860-718a-42fc-af92-d0466a2f51fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:13.125188 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96b2860-718a-42fc-af92-d0466a2f51fc service neutron] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:13.125474 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96b2860-718a-42fc-af92-d0466a2f51fc service neutron] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:13.125750 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96b2860-718a-42fc-af92-d0466a2f51fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:13.137846 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96b2860-718a-42fc-af92-d0466a2f51fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:13.139607 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96b2860-718a-42fc-af92-d0466a2f51fc service neutron] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:13.168973 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 989/1996] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:20:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:20:13.176551 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96b2860-718a-42fc-af92-d0466a2f51fc service neutron] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:13.197922 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96b2860-718a-42fc-af92-d0466a2f51fc service neutron] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:13.204409 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3ea42a-6cbd-40a2-a960-5e14b30f8c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:13.206146 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3ea42a-6cbd-40a2-a960-5e14b30f8c20 admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:13.206497 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3ea42a-6cbd-40a2-a960-5e14b30f8c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:13.206837 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3ea42a-6cbd-40a2-a960-5e14b30f8c20 admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:13.207607 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3ea42a-6cbd-40a2-a960-5e14b30f8c20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:13.209558 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3ea42a-6cbd-40a2-a960-5e14b30f8c20 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:13.225017 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 990/1997] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:20:13 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:20:13.230696 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3ea42a-6cbd-40a2-a960-5e14b30f8c20 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:13.245869 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1008/1998] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:20:13 2022] POST /identity/v3/users => generated 299 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:20:13.252857 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb7ae75-68e2-4f7e-a5e1-9a19da013733 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:13.265589 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3ea42a-6cbd-40a2-a960-5e14b30f8c20 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:13.274905 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb7ae75-68e2-4f7e-a5e1-9a19da013733 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:13.276900 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a8093a-0983-4bba-9921-55a766f3e651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:13.278234 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a8093a-0983-4bba-9921-55a766f3e651 admin admin] REQUEST_METHOD: `PUT` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:13.278234 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a8093a-0983-4bba-9921-55a766f3e651 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:13.278234 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a8093a-0983-4bba-9921-55a766f3e651 admin admin] PATH_INFO: `/v3/projects/85eac120275644fab4c388555a03fa70/users/2e5300071df743409447c56951ddc25b/roles/3255f77192524a16bc405173106871fc` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:13.286016 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f064fa8-d2fb-4eb9-8dc4-c5dd2abb96ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:13.287321 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f064fa8-d2fb-4eb9-8dc4-c5dd2abb96ce admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:13.287321 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f064fa8-d2fb-4eb9-8dc4-c5dd2abb96ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:13.287321 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f064fa8-d2fb-4eb9-8dc4-c5dd2abb96ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:13.287950 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f064fa8-d2fb-4eb9-8dc4-c5dd2abb96ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:13.289682 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f064fa8-d2fb-4eb9-8dc4-c5dd2abb96ce admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:13.314853 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a8093a-0983-4bba-9921-55a766f3e651 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85eac120275644fab4c388555a03fa70, user_id=2e5300071df743409447c56951ddc25b, role_id=3255f77192524a16bc405173106871fc)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:13.317553 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a8093a-0983-4bba-9921-55a766f3e651 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:13.318327 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1009/1999] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:20:13 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:20:13.323047 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f064fa8-d2fb-4eb9-8dc4-c5dd2abb96ce admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:13.340823 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f064fa8-d2fb-4eb9-8dc4-c5dd2abb96ce admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:13.348329 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b934f0-ef93-4f4b-9743-a52d00a8b0cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:13.349472 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b934f0-ef93-4f4b-9743-a52d00a8b0cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:13.349981 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b934f0-ef93-4f4b-9743-a52d00a8b0cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:13.350509 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b934f0-ef93-4f4b-9743-a52d00a8b0cb admin admin] PATH_INFO: `/v3/projects/5c3999fd375745c29e8dbd72a22fa5dc/users/1196f51d38e043219bfad6215fe71004/roles/62f19ea272b447ccaab1576cd9bf4fd9` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:13.370357 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b934f0-ef93-4f4b-9743-a52d00a8b0cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c3999fd375745c29e8dbd72a22fa5dc, user_id=1196f51d38e043219bfad6215fe71004, role_id=62f19ea272b447ccaab1576cd9bf4fd9)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:13.373465 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b934f0-ef93-4f4b-9743-a52d00a8b0cb admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:13.415769 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 991/2000] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:20:13 2022] PUT /identity/v3/projects/85eac120275644fab4c388555a03fa70/users/2e5300071df743409447c56951ddc25b/roles/3255f77192524a16bc405173106871fc => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:20:13.426595 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fc6645-fad7-40eb-aca7-bcd88e0062d2 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:13.426885 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fc6645-fad7-40eb-aca7-bcd88e0062d2 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:13.427174 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fc6645-fad7-40eb-aca7-bcd88e0062d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:13.459677 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-f9fc6645-fad7-40eb-aca7-bcd88e0062d2 None None] MFA Rules not processed for user `2e5300071df743409447c56951ddc25b`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:20:13.460933 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-f9fc6645-fad7-40eb-aca7-bcd88e0062d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:13.527135 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1010/2001] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:20:13 2022] PUT /identity/v3/projects/5c3999fd375745c29e8dbd72a22fa5dc/users/1196f51d38e043219bfad6215fe71004/roles/62f19ea272b447ccaab1576cd9bf4fd9 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:20:13.532523 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b934f0-ef93-4f4b-9743-a52d00a8b0cb admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:13.546426 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 992/2002] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:20:13 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:20:13.551141 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fc6645-fad7-40eb-aca7-bcd88e0062d2 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:13.608889 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b934f0-ef93-4f4b-9743-a52d00a8b0cb admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:13.614370 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e9ed23-7462-4554-9c50-b0da5893b61d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:13.614994 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e9ed23-7462-4554-9c50-b0da5893b61d admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:13.615277 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e9ed23-7462-4554-9c50-b0da5893b61d admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:13.615553 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e9ed23-7462-4554-9c50-b0da5893b61d admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:13.616243 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e9ed23-7462-4554-9c50-b0da5893b61d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:13.618006 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e9ed23-7462-4554-9c50-b0da5893b61d admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:13.627892 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fc6645-fad7-40eb-aca7-bcd88e0062d2 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:13.628175 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1011/2003] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:20:13 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 25 03:20:13.632521 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e9ed23-7462-4554-9c50-b0da5893b61d admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:13.633459 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96764773-6b3a-453c-86b5-565730f22101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:13.634086 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96764773-6b3a-453c-86b5-565730f22101 admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:13.634400 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96764773-6b3a-453c-86b5-565730f22101 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:13.634682 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96764773-6b3a-453c-86b5-565730f22101 admin admin] PATH_INFO: `/v3/projects` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:13.635306 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96764773-6b3a-453c-86b5-565730f22101 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:13.637228 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96764773-6b3a-453c-86b5-565730f22101 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:13.643243 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e9ed23-7462-4554-9c50-b0da5893b61d admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:13.647921 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff59fe76-8d32-422a-b1c9-c417ade2d704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:13.648578 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff59fe76-8d32-422a-b1c9-c417ade2d704 admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:13.648862 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff59fe76-8d32-422a-b1c9-c417ade2d704 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:13.649137 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff59fe76-8d32-422a-b1c9-c417ade2d704 admin admin] PATH_INFO: `/v3/projects/5c3999fd375745c29e8dbd72a22fa5dc/users/1196f51d38e043219bfad6215fe71004/roles/62f19ea272b447ccaab1576cd9bf4fd9` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:13.651107 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff59fe76-8d32-422a-b1c9-c417ade2d704 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c3999fd375745c29e8dbd72a22fa5dc, user_id=1196f51d38e043219bfad6215fe71004, role_id=62f19ea272b447ccaab1576cd9bf4fd9)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:13.652195 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:20:13.653739 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff59fe76-8d32-422a-b1c9-c417ade2d704 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:13.664777 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1012/2004] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:20:13 2022] PUT /identity/v3/projects/5c3999fd375745c29e8dbd72a22fa5dc/users/1196f51d38e043219bfad6215fe71004/roles/62f19ea272b447ccaab1576cd9bf4fd9 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:20:13.675511 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4b3a55-a337-4415-9049-9cb21c6311d3 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:13.676008 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4b3a55-a337-4415-9049-9cb21c6311d3 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:13.676110 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4b3a55-a337-4415-9049-9cb21c6311d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:13.705163 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-4c4b3a55-a337-4415-9049-9cb21c6311d3 None None] MFA Rules not processed for user `1196f51d38e043219bfad6215fe71004`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:20:13.706456 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-4c4b3a55-a337-4415-9049-9cb21c6311d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:13.733312 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 993/2005] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:20:13 2022] POST /identity/v3/projects => generated 383 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:20:13.733312 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96764773-6b3a-453c-86b5-565730f22101 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:13.733482 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:20:13.733558 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-96764773-6b3a-453c-86b5-565730f22101 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:13.783308 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1013/2006] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:20:13 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:20:13.784189 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4b3a55-a337-4415-9049-9cb21c6311d3 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:13.801111 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96764773-6b3a-453c-86b5-565730f22101 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:13.805833 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61933bb7-bb1a-404f-bb58-e56a0f5e7007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:13.806412 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61933bb7-bb1a-404f-bb58-e56a0f5e7007 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:13.806673 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61933bb7-bb1a-404f-bb58-e56a0f5e7007 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:13.807244 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61933bb7-bb1a-404f-bb58-e56a0f5e7007 admin admin] PATH_INFO: `/v3/projects/94a25b21191341f39080554fdfe55e65` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:13.808099 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61933bb7-bb1a-404f-bb58-e56a0f5e7007 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94a25b21191341f39080554fdfe55e65)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:13.810052 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61933bb7-bb1a-404f-bb58-e56a0f5e7007 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:13.857473 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4b3a55-a337-4415-9049-9cb21c6311d3 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:13.862813 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3df2f7-97cc-4f6b-b90e-ad7ffc62501b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:13.863429 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3df2f7-97cc-4f6b-b90e-ad7ffc62501b admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:13.863722 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3df2f7-97cc-4f6b-b90e-ad7ffc62501b admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:13.863997 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3df2f7-97cc-4f6b-b90e-ad7ffc62501b admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:13.864633 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3df2f7-97cc-4f6b-b90e-ad7ffc62501b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:13.866597 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3df2f7-97cc-4f6b-b90e-ad7ffc62501b admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:13.956776 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1014/2007] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:20:13 2022] POST /identity/v3/users => generated 298 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:20:13.961087 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3df2f7-97cc-4f6b-b90e-ad7ffc62501b admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:13.963062 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-6b3df2f7-97cc-4f6b-b90e-ad7ffc62501b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:13.976821 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-61933bb7-bb1a-404f-bb58-e56a0f5e7007 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94a25b21191341f39080554fdfe55e65', 'request_id': 'req-61933bb7-bb1a-404f-bb58-e56a0f5e7007'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:14.029568 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3df2f7-97cc-4f6b-b90e-ad7ffc62501b admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:14.085312 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11efb96a-6418-4d13-9edf-96e6180e4162 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60193ba9e1c14df3bc9eebcc6f28f2d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:14.085981 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11efb96a-6418-4d13-9edf-96e6180e4162 service cinder] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:14.086201 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11efb96a-6418-4d13-9edf-96e6180e4162 service cinder] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:14.086513 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11efb96a-6418-4d13-9edf-96e6180e4162 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:14.105312 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:20:14.132262 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-61933bb7-bb1a-404f-bb58-e56a0f5e7007 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94a25b21191341f39080554fdfe55e65', 'request_id': 'req-61933bb7-bb1a-404f-bb58-e56a0f5e7007'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:14.140561 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-61933bb7-bb1a-404f-bb58-e56a0f5e7007 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94a25b21191341f39080554fdfe55e65', 'request_id': 'req-61933bb7-bb1a-404f-bb58-e56a0f5e7007'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:14.179735 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11efb96a-6418-4d13-9edf-96e6180e4162 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:14.182477 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11efb96a-6418-4d13-9edf-96e6180e4162 service cinder] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:14.184393 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-61933bb7-bb1a-404f-bb58-e56a0f5e7007 admin admin] The token cache is being invalidate because project 94a25b21191341f39080554fdfe55e65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81995) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:20:14.185012 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-61933bb7-bb1a-404f-bb58-e56a0f5e7007 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:14.186269 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-61933bb7-bb1a-404f-bb58-e56a0f5e7007 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:14.188151 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 994/2008] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:20:13 2022] DELETE /identity/v3/projects/94a25b21191341f39080554fdfe55e65 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:20:14.189193 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61933bb7-bb1a-404f-bb58-e56a0f5e7007 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:14.190670 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-61933bb7-bb1a-404f-bb58-e56a0f5e7007 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:14.215252 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1015/2009] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:20:13 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:20:14.216890 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11efb96a-6418-4d13-9edf-96e6180e4162 service cinder] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:14.308013 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61933bb7-bb1a-404f-bb58-e56a0f5e7007 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:14.344425 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85ee7b3-6a4c-4bbc-aa23-18f8a87da9f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:14.345046 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85ee7b3-6a4c-4bbc-aa23-18f8a87da9f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:14.345363 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85ee7b3-6a4c-4bbc-aa23-18f8a87da9f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:14.345641 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85ee7b3-6a4c-4bbc-aa23-18f8a87da9f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:14.346331 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85ee7b3-6a4c-4bbc-aa23-18f8a87da9f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:14.347957 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85ee7b3-6a4c-4bbc-aa23-18f8a87da9f6 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:14.363123 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11efb96a-6418-4d13-9edf-96e6180e4162 service cinder] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:14.365675 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 995/2010] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:20:14 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:20:14.373965 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85ee7b3-6a4c-4bbc-aa23-18f8a87da9f6 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:14.385763 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87408d2d-1e2b-4816-9f3f-c945b6bcc9a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:14.386546 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87408d2d-1e2b-4816-9f3f-c945b6bcc9a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:14.386878 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87408d2d-1e2b-4816-9f3f-c945b6bcc9a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:14.393212 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87408d2d-1e2b-4816-9f3f-c945b6bcc9a6 admin admin] PATH_INFO: `/v3/users/bb13f202fabd4ffe838fceecb6613b70` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:14.393212 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87408d2d-1e2b-4816-9f3f-c945b6bcc9a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb13f202fabd4ffe838fceecb6613b70)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:14.393212 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87408d2d-1e2b-4816-9f3f-c945b6bcc9a6 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:14.398829 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85ee7b3-6a4c-4bbc-aa23-18f8a87da9f6 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:14.413551 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dc10f5-8207-4ffb-9a98-1b5d46416c92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:14.414483 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dc10f5-8207-4ffb-9a98-1b5d46416c92 admin admin] REQUEST_METHOD: `PUT` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:14.414834 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dc10f5-8207-4ffb-9a98-1b5d46416c92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:14.415177 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dc10f5-8207-4ffb-9a98-1b5d46416c92 admin admin] PATH_INFO: `/v3/projects/506a9ad77ba8400a889b0d0527f658fd/users/eaa083869df34ff1a153981df0ee1143/roles/62f19ea272b447ccaab1576cd9bf4fd9` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:14.442341 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dc10f5-8207-4ffb-9a98-1b5d46416c92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=506a9ad77ba8400a889b0d0527f658fd, user_id=eaa083869df34ff1a153981df0ee1143, role_id=62f19ea272b447ccaab1576cd9bf4fd9)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:14.449791 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dc10f5-8207-4ffb-9a98-1b5d46416c92 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:14.461901 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:INSERT=3,keystone:SELECT=90,keystone:DELETE=7 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:20:14.569753 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 996/2011] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:20:14 2022] PUT /identity/v3/projects/506a9ad77ba8400a889b0d0527f658fd/users/eaa083869df34ff1a153981df0ee1143/roles/62f19ea272b447ccaab1576cd9bf4fd9 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:20:14.572479 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dc10f5-8207-4ffb-9a98-1b5d46416c92 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:14.646821 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-87408d2d-1e2b-4816-9f3f-c945b6bcc9a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb13f202fabd4ffe838fceecb6613b70'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:14.652369 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dc10f5-8207-4ffb-9a98-1b5d46416c92 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:14.657525 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631e95d8-9448-4299-aeb2-4287fc2502e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:14.658856 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631e95d8-9448-4299-aeb2-4287fc2502e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:14.659289 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631e95d8-9448-4299-aeb2-4287fc2502e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:14.659631 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631e95d8-9448-4299-aeb2-4287fc2502e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:14.660293 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631e95d8-9448-4299-aeb2-4287fc2502e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:14.661915 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631e95d8-9448-4299-aeb2-4287fc2502e3 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:14.671721 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 997/2012] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:20:14 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 25 03:20:14.676902 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631e95d8-9448-4299-aeb2-4287fc2502e3 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:14.686050 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631e95d8-9448-4299-aeb2-4287fc2502e3 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:14.691538 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe07c3ba-e1e3-47b0-8b0e-2556d1bd506f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:14.692196 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe07c3ba-e1e3-47b0-8b0e-2556d1bd506f admin admin] REQUEST_METHOD: `PUT` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:14.692478 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe07c3ba-e1e3-47b0-8b0e-2556d1bd506f admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:14.692749 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe07c3ba-e1e3-47b0-8b0e-2556d1bd506f admin admin] PATH_INFO: `/v3/projects/506a9ad77ba8400a889b0d0527f658fd/users/eaa083869df34ff1a153981df0ee1143/roles/62f19ea272b447ccaab1576cd9bf4fd9` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:14.694821 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe07c3ba-e1e3-47b0-8b0e-2556d1bd506f admin admin] RBAC: Authorizing `identity:create_grant(project_id=506a9ad77ba8400a889b0d0527f658fd, user_id=eaa083869df34ff1a153981df0ee1143, role_id=62f19ea272b447ccaab1576cd9bf4fd9)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:14.697498 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe07c3ba-e1e3-47b0-8b0e-2556d1bd506f admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:14.708106 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 998/2013] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:20:14 2022] PUT /identity/v3/projects/506a9ad77ba8400a889b0d0527f658fd/users/eaa083869df34ff1a153981df0ee1143/roles/62f19ea272b447ccaab1576cd9bf4fd9 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:20:14.719317 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3995314-c47f-428b-8e35-013fccf044e3 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:14.719693 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3995314-c47f-428b-8e35-013fccf044e3 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:14.719972 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3995314-c47f-428b-8e35-013fccf044e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:14.743539 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-87408d2d-1e2b-4816-9f3f-c945b6bcc9a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb13f202fabd4ffe838fceecb6613b70'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:14.744436 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-87408d2d-1e2b-4816-9f3f-c945b6bcc9a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb13f202fabd4ffe838fceecb6613b70'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:14.752476 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-f3995314-c47f-428b-8e35-013fccf044e3 None None] MFA Rules not processed for user `eaa083869df34ff1a153981df0ee1143`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:20:14.755151 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-f3995314-c47f-428b-8e35-013fccf044e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:14.756444 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-87408d2d-1e2b-4816-9f3f-c945b6bcc9a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb13f202fabd4ffe838fceecb6613b70'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:14.771975 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-87408d2d-1e2b-4816-9f3f-c945b6bcc9a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb13f202fabd4ffe838fceecb6613b70'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:14.778927 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1016/2014] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:20:14 2022] DELETE /identity/v3/users/bb13f202fabd4ffe838fceecb6613b70 => generated 0 bytes in 563 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:20:14.842576 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 999/2015] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:20:14 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:20:15.008348 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87408d2d-1e2b-4816-9f3f-c945b6bcc9a6 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:15.009878 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-87408d2d-1e2b-4816-9f3f-c945b6bcc9a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:15.072676 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:20:15.092280 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87408d2d-1e2b-4816-9f3f-c945b6bcc9a6 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:15.097641 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2850ef9c-1485-4f76-92f9-8c4492a71c9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60193ba9e1c14df3bc9eebcc6f28f2d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:15.098540 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2850ef9c-1485-4f76-92f9-8c4492a71c9e service cinder] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:15.098823 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2850ef9c-1485-4f76-92f9-8c4492a71c9e service cinder] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:15.099098 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2850ef9c-1485-4f76-92f9-8c4492a71c9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:15.163772 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2850ef9c-1485-4f76-92f9-8c4492a71c9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:15.165234 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2850ef9c-1485-4f76-92f9-8c4492a71c9e service cinder] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:15.175264 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1017/2016] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:20:15 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:20:15.234228 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3995314-c47f-428b-8e35-013fccf044e3 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:15.236419 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-f3995314-c47f-428b-8e35-013fccf044e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:15.316934 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3995314-c47f-428b-8e35-013fccf044e3 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:15.325231 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:20:15.398311 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde5b029-8215-4180-9a2a-69ad0c2fb9d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:15.398973 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde5b029-8215-4180-9a2a-69ad0c2fb9d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:15.399313 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde5b029-8215-4180-9a2a-69ad0c2fb9d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:15.399598 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde5b029-8215-4180-9a2a-69ad0c2fb9d0 admin admin] PATH_INFO: `/v3/projects/0c0fc58f428c44e6a8c8556917a38403` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:15.400784 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde5b029-8215-4180-9a2a-69ad0c2fb9d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c0fc58f428c44e6a8c8556917a38403)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:15.403091 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde5b029-8215-4180-9a2a-69ad0c2fb9d0 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:15.437360 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2850ef9c-1485-4f76-92f9-8c4492a71c9e service cinder] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:15.446940 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2850ef9c-1485-4f76-92f9-8c4492a71c9e service cinder] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:15.451139 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caafdd4-d465-49fd-abff-d8d2a2281a0d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60193ba9e1c14df3bc9eebcc6f28f2d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:15.451824 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caafdd4-d465-49fd-abff-d8d2a2281a0d service cinder] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:15.452200 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caafdd4-d465-49fd-abff-d8d2a2281a0d service cinder] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:15.452542 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caafdd4-d465-49fd-abff-d8d2a2281a0d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:15.454368 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-3caafdd4-d465-49fd-abff-d8d2a2281a0d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:15.507441 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-fde5b029-8215-4180-9a2a-69ad0c2fb9d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c0fc58f428c44e6a8c8556917a38403', 'request_id': 'req-fde5b029-8215-4180-9a2a-69ad0c2fb9d0'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:15.530744 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caafdd4-d465-49fd-abff-d8d2a2281a0d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:15.532659 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caafdd4-d465-49fd-abff-d8d2a2281a0d service cinder] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:15.542822 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1018/2017] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:20:15 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:20:15.559929 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-fde5b029-8215-4180-9a2a-69ad0c2fb9d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c0fc58f428c44e6a8c8556917a38403', 'request_id': 'req-fde5b029-8215-4180-9a2a-69ad0c2fb9d0'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:15.567051 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-fde5b029-8215-4180-9a2a-69ad0c2fb9d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c0fc58f428c44e6a8c8556917a38403', 'request_id': 'req-fde5b029-8215-4180-9a2a-69ad0c2fb9d0'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:15.586580 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-fde5b029-8215-4180-9a2a-69ad0c2fb9d0 admin admin] The token cache is being invalidate because project 0c0fc58f428c44e6a8c8556917a38403 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81995) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:20:15.586979 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-fde5b029-8215-4180-9a2a-69ad0c2fb9d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:15.589991 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-fde5b029-8215-4180-9a2a-69ad0c2fb9d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:15.591927 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1000/2018] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:20:15 2022] DELETE /identity/v3/projects/0c0fc58f428c44e6a8c8556917a38403 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:20:15.596423 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0204a6-8cb2-468e-806b-a346da7c37c0 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:15.596799 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0204a6-8cb2-468e-806b-a346da7c37c0 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:15.597113 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0204a6-8cb2-468e-806b-a346da7c37c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:15.619115 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-af0204a6-8cb2-468e-806b-a346da7c37c0 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:20:15.620243 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-af0204a6-8cb2-468e-806b-a346da7c37c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:15.709466 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1019/2019] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:20:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:20:15.718491 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21d1352-78fb-47b4-b981-bf362b8bfde1 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:15.719549 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21d1352-78fb-47b4-b981-bf362b8bfde1 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:15.719911 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21d1352-78fb-47b4-b981-bf362b8bfde1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:15.754712 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-a21d1352-78fb-47b4-b981-bf362b8bfde1 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:20:15.759917 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-a21d1352-78fb-47b4-b981-bf362b8bfde1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:15.774885 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1001/2020] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:20:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:20:15.779206 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0204a6-8cb2-468e-806b-a346da7c37c0 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:15.790783 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0204a6-8cb2-468e-806b-a346da7c37c0 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:15.796577 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2616563f-acc4-4dbe-86f9-f2e2e4e6180f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:15.798159 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2616563f-acc4-4dbe-86f9-f2e2e4e6180f admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:15.798159 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2616563f-acc4-4dbe-86f9-f2e2e4e6180f admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:15.798159 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2616563f-acc4-4dbe-86f9-f2e2e4e6180f admin admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:15.799344 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2616563f-acc4-4dbe-86f9-f2e2e4e6180f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:20:15.799655 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2616563f-acc4-4dbe-86f9-f2e2e4e6180f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:15.801518 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2616563f-acc4-4dbe-86f9-f2e2e4e6180f admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:15.814545 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1020/2021] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Aug 25 03:20:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:20:15.819138 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21d1352-78fb-47b4-b981-bf362b8bfde1 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:15.832744 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21d1352-78fb-47b4-b981-bf362b8bfde1 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:15.837437 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a01fe1-0e0c-4d33-95ce-58c275809822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:15.838045 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a01fe1-0e0c-4d33-95ce-58c275809822 admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:15.838380 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a01fe1-0e0c-4d33-95ce-58c275809822 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:15.838650 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a01fe1-0e0c-4d33-95ce-58c275809822 admin admin] PATH_INFO: `/v3/projects` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:15.839271 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a01fe1-0e0c-4d33-95ce-58c275809822 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:15.841195 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a01fe1-0e0c-4d33-95ce-58c275809822 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:15.935188 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1002/2022] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:20:15 2022] POST /identity/v3/projects => generated 361 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:20:15.939819 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2616563f-acc4-4dbe-86f9-f2e2e4e6180f admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:15.949936 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2616563f-acc4-4dbe-86f9-f2e2e4e6180f admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:15.958055 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:20:16.007218 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5b0b22-e269-49b1-8368-41181b7919d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:16.007798 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5b0b22-e269-49b1-8368-41181b7919d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:16.008076 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5b0b22-e269-49b1-8368-41181b7919d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:16.008346 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5b0b22-e269-49b1-8368-41181b7919d3 admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:16.010090 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5b0b22-e269-49b1-8368-41181b7919d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:16.011052 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5b0b22-e269-49b1-8368-41181b7919d3 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:16.051530 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a01fe1-0e0c-4d33-95ce-58c275809822 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:16.053245 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-f4a01fe1-0e0c-4d33-95ce-58c275809822 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:16.065555 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a01fe1-0e0c-4d33-95ce-58c275809822 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:16.070157 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bad31a-d631-4388-9b67-1ed1bf587841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:16.070837 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bad31a-d631-4388-9b67-1ed1bf587841 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:16.071115 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bad31a-d631-4388-9b67-1ed1bf587841 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:16.071386 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bad31a-d631-4388-9b67-1ed1bf587841 admin admin] PATH_INFO: `/v3/users/2e5300071df743409447c56951ddc25b` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:16.072506 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bad31a-d631-4388-9b67-1ed1bf587841 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e5300071df743409447c56951ddc25b)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:16.074721 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bad31a-d631-4388-9b67-1ed1bf587841 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:16.084169 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1021/2023] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:20:15 2022] POST /identity/v3/users => generated 281 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:20:16.085161 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5b0b22-e269-49b1-8368-41181b7919d3 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:16.096355 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5b0b22-e269-49b1-8368-41181b7919d3 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:16.101030 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0425c138-d349-49db-b3e2-da212cec2590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:16.101632 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0425c138-d349-49db-b3e2-da212cec2590 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:16.101918 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0425c138-d349-49db-b3e2-da212cec2590 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:16.102198 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0425c138-d349-49db-b3e2-da212cec2590 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:16.102995 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0425c138-d349-49db-b3e2-da212cec2590 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:16.104633 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0425c138-d349-49db-b3e2-da212cec2590 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:16.115960 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1022/2024] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:20:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 25 03:20:16.119769 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0425c138-d349-49db-b3e2-da212cec2590 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:16.129113 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0425c138-d349-49db-b3e2-da212cec2590 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:16.133728 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b39ea95-91f2-42c3-b52a-84dd271f2f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:16.134324 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b39ea95-91f2-42c3-b52a-84dd271f2f99 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:16.134603 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b39ea95-91f2-42c3-b52a-84dd271f2f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:16.134879 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b39ea95-91f2-42c3-b52a-84dd271f2f99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:16.135500 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b39ea95-91f2-42c3-b52a-84dd271f2f99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:16.141449 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b39ea95-91f2-42c3-b52a-84dd271f2f99 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:16.146539 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1023/2025] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:20:16 2022] GET /identity/v3/roles => generated 1420 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 25 03:20:16.150315 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b39ea95-91f2-42c3-b52a-84dd271f2f99 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:16.159949 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b39ea95-91f2-42c3-b52a-84dd271f2f99 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:16.164629 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682f65a0-cd54-4a1d-bb8f-c633ed3bf3ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:16.165255 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682f65a0-cd54-4a1d-bb8f-c633ed3bf3ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:16.165578 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682f65a0-cd54-4a1d-bb8f-c633ed3bf3ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:16.165822 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682f65a0-cd54-4a1d-bb8f-c633ed3bf3ce admin admin] PATH_INFO: `/v3/projects/fe407cc5a52940c585bb1d2d9efd7c07/users/16ba7682df534212ace9509688c0c183/roles/3255f77192524a16bc405173106871fc` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:16.180560 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682f65a0-cd54-4a1d-bb8f-c633ed3bf3ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe407cc5a52940c585bb1d2d9efd7c07, user_id=16ba7682df534212ace9509688c0c183, role_id=3255f77192524a16bc405173106871fc)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:16.183250 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682f65a0-cd54-4a1d-bb8f-c633ed3bf3ce admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:16.287783 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1024/2026] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:20:16 2022] PUT /identity/v3/projects/fe407cc5a52940c585bb1d2d9efd7c07/users/16ba7682df534212ace9509688c0c183/roles/3255f77192524a16bc405173106871fc => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:20:16.299760 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-f3bad31a-d631-4388-9b67-1ed1bf587841 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e5300071df743409447c56951ddc25b'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:16.305893 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e5efd8-9b39-4c72-b25e-87420545aabb None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:16.306316 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e5efd8-9b39-4c72-b25e-87420545aabb None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:16.306681 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e5efd8-9b39-4c72-b25e-87420545aabb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:16.334734 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-89e5efd8-9b39-4c72-b25e-87420545aabb None None] MFA Rules not processed for user `16ba7682df534212ace9509688c0c183`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:20:16.335966 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-89e5efd8-9b39-4c72-b25e-87420545aabb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:16.377468 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-f3bad31a-d631-4388-9b67-1ed1bf587841 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e5300071df743409447c56951ddc25b'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:16.378114 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-f3bad31a-d631-4388-9b67-1ed1bf587841 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e5300071df743409447c56951ddc25b'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:16.378770 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-f3bad31a-d631-4388-9b67-1ed1bf587841 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e5300071df743409447c56951ddc25b'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:16.390893 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=81,keystone:DELETE=16,keystone:INSERT=3 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:20:16.394019 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-f3bad31a-d631-4388-9b67-1ed1bf587841 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e5300071df743409447c56951ddc25b'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:16.408445 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1003/2027] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:20:16 2022] DELETE /identity/v3/users/2e5300071df743409447c56951ddc25b => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:20:16.408912 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1025/2028] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:20:16 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:20:16.412941 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c09126d-fa9b-4f31-80b1-5662141b1070 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:16.413259 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c09126d-fa9b-4f31-80b1-5662141b1070 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:16.413542 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c09126d-fa9b-4f31-80b1-5662141b1070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:16.415068 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e5efd8-9b39-4c72-b25e-87420545aabb None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:16.416759 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-89e5efd8-9b39-4c72-b25e-87420545aabb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:16.441229 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-4c09126d-fa9b-4f31-80b1-5662141b1070 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:20:16.442633 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-4c09126d-fa9b-4f31-80b1-5662141b1070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:16.470971 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:20:16.491449 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e5efd8-9b39-4c72-b25e-87420545aabb None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:16.508667 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1004/2029] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:20:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:20:16.518942 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c09126d-fa9b-4f31-80b1-5662141b1070 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:16.520945 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-4c09126d-fa9b-4f31-80b1-5662141b1070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:16.549844 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09d5251-2cb4-4c98-9270-5091a446e6fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:16.550628 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09d5251-2cb4-4c98-9270-5091a446e6fb admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:16.550915 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09d5251-2cb4-4c98-9270-5091a446e6fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:16.551197 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09d5251-2cb4-4c98-9270-5091a446e6fb admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:16.552215 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09d5251-2cb4-4c98-9270-5091a446e6fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:16.553765 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09d5251-2cb4-4c98-9270-5091a446e6fb admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:16.578502 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c09126d-fa9b-4f31-80b1-5662141b1070 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:16.582921 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98e10f4-1e7b-476b-a5d4-9d61c87e53d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:16.583622 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98e10f4-1e7b-476b-a5d4-9d61c87e53d2 service neutron] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:16.583982 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98e10f4-1e7b-476b-a5d4-9d61c87e53d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:16.584321 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98e10f4-1e7b-476b-a5d4-9d61c87e53d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:16.594558 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98e10f4-1e7b-476b-a5d4-9d61c87e53d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:16.596057 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98e10f4-1e7b-476b-a5d4-9d61c87e53d2 service neutron] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:16.605889 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1005/2030] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:20:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 25 03:20:16.648229 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1026/2031] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:20:16 2022] POST /identity/v3/projects => generated 365 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:20:16.652361 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98e10f4-1e7b-476b-a5d4-9d61c87e53d2 service neutron] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:16.664636 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98e10f4-1e7b-476b-a5d4-9d61c87e53d2 service neutron] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:16.735870 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc71e60-7a8d-4a30-9944-e00e58e1b3d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:16.736786 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc71e60-7a8d-4a30-9944-e00e58e1b3d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:16.737161 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc71e60-7a8d-4a30-9944-e00e58e1b3d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:16.737532 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc71e60-7a8d-4a30-9944-e00e58e1b3d3 admin admin] PATH_INFO: `/v3/users` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:16.738260 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc71e60-7a8d-4a30-9944-e00e58e1b3d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:16.740384 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc71e60-7a8d-4a30-9944-e00e58e1b3d3 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:16.830857 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1006/2032] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:20:16 2022] POST /identity/v3/users => generated 289 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:20:16.836719 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09d5251-2cb4-4c98-9270-5091a446e6fb admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:16.847278 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09d5251-2cb4-4c98-9270-5091a446e6fb admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:16.856636 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb77f331-a61a-4eb7-80e8-98722dab2847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:16.857434 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb77f331-a61a-4eb7-80e8-98722dab2847 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:16.857812 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb77f331-a61a-4eb7-80e8-98722dab2847 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:16.858526 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb77f331-a61a-4eb7-80e8-98722dab2847 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:16.859877 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb77f331-a61a-4eb7-80e8-98722dab2847 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:16.861726 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb77f331-a61a-4eb7-80e8-98722dab2847 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:16.873576 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1027/2033] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:20:16 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:20:16.878453 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc71e60-7a8d-4a30-9944-e00e58e1b3d3 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:16.888659 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc71e60-7a8d-4a30-9944-e00e58e1b3d3 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:16.894257 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48a9e48-d33c-497f-9c16-0508b658f48f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:16.894978 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48a9e48-d33c-497f-9c16-0508b658f48f admin admin] REQUEST_METHOD: `PUT` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:16.897369 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48a9e48-d33c-497f-9c16-0508b658f48f admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:16.897369 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48a9e48-d33c-497f-9c16-0508b658f48f admin admin] PATH_INFO: `/v3/projects/2b56f28119b749acaae9738160cd5406/users/1e3d14da2c5a4e4886f7d28dcc0913f3/roles/62f19ea272b447ccaab1576cd9bf4fd9` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:16.913807 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48a9e48-d33c-497f-9c16-0508b658f48f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b56f28119b749acaae9738160cd5406, user_id=1e3d14da2c5a4e4886f7d28dcc0913f3, role_id=62f19ea272b447ccaab1576cd9bf4fd9)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:16.926366 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48a9e48-d33c-497f-9c16-0508b658f48f admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:16.926366 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:20:17.001316 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1007/2034] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:20:16 2022] PUT /identity/v3/projects/2b56f28119b749acaae9738160cd5406/users/1e3d14da2c5a4e4886f7d28dcc0913f3/roles/62f19ea272b447ccaab1576cd9bf4fd9 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:20:17.005681 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb77f331-a61a-4eb7-80e8-98722dab2847 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:17.016662 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb77f331-a61a-4eb7-80e8-98722dab2847 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:17.080135 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ad6dbe-3e5f-4593-87e3-3ba976f79955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:17.080752 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ad6dbe-3e5f-4593-87e3-3ba976f79955 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:17.081034 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ad6dbe-3e5f-4593-87e3-3ba976f79955 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:17.081312 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ad6dbe-3e5f-4593-87e3-3ba976f79955 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:17.081956 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ad6dbe-3e5f-4593-87e3-3ba976f79955 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:17.083583 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ad6dbe-3e5f-4593-87e3-3ba976f79955 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:17.093507 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1028/2035] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:20:17 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:20:17.097634 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48a9e48-d33c-497f-9c16-0508b658f48f admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:17.108124 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48a9e48-d33c-497f-9c16-0508b658f48f admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:17.113825 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fa5c2c-b1d9-4067-9047-d57f054f523d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:17.114506 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fa5c2c-b1d9-4067-9047-d57f054f523d admin admin] REQUEST_METHOD: `PUT` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:17.114786 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fa5c2c-b1d9-4067-9047-d57f054f523d admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:17.115058 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fa5c2c-b1d9-4067-9047-d57f054f523d admin admin] PATH_INFO: `/v3/projects/2b56f28119b749acaae9738160cd5406/users/1e3d14da2c5a4e4886f7d28dcc0913f3/roles/62f19ea272b447ccaab1576cd9bf4fd9` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:17.117038 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fa5c2c-b1d9-4067-9047-d57f054f523d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b56f28119b749acaae9738160cd5406, user_id=1e3d14da2c5a4e4886f7d28dcc0913f3, role_id=62f19ea272b447ccaab1576cd9bf4fd9)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:17.119711 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fa5c2c-b1d9-4067-9047-d57f054f523d admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:17.130798 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1008/2036] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:20:17 2022] PUT /identity/v3/projects/2b56f28119b749acaae9738160cd5406/users/1e3d14da2c5a4e4886f7d28dcc0913f3/roles/62f19ea272b447ccaab1576cd9bf4fd9 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:20:17.141773 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424c005f-f788-46dc-ac1c-a78a4d5db0ff None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:17.142080 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424c005f-f788-46dc-ac1c-a78a4d5db0ff None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:17.142392 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424c005f-f788-46dc-ac1c-a78a4d5db0ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:17.145064 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fa5c2c-b1d9-4067-9047-d57f054f523d admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:17.147606 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-20fa5c2c-b1d9-4067-9047-d57f054f523d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:17.179885 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-424c005f-f788-46dc-ac1c-a78a4d5db0ff None None] MFA Rules not processed for user `1e3d14da2c5a4e4886f7d28dcc0913f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:20:17.179885 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-424c005f-f788-46dc-ac1c-a78a4d5db0ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:17.220880 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fa5c2c-b1d9-4067-9047-d57f054f523d admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:17.225213 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:20:17.225889 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050426cf-5f37-4171-a00d-013c6b2543f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:17.226246 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050426cf-5f37-4171-a00d-013c6b2543f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:17.226674 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050426cf-5f37-4171-a00d-013c6b2543f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:17.226995 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050426cf-5f37-4171-a00d-013c6b2543f0 admin admin] PATH_INFO: `/v3/projects/85eac120275644fab4c388555a03fa70` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:17.228113 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050426cf-5f37-4171-a00d-013c6b2543f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=85eac120275644fab4c388555a03fa70)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:17.231011 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050426cf-5f37-4171-a00d-013c6b2543f0 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:17.245841 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1029/2037] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:20:17 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:20:17.259298 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424c005f-f788-46dc-ac1c-a78a4d5db0ff None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:17.260675 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-424c005f-f788-46dc-ac1c-a78a4d5db0ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:17.317011 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424c005f-f788-46dc-ac1c-a78a4d5db0ff None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:17.320704 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8ad25f-c336-4726-8086-11a80f16ac82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60193ba9e1c14df3bc9eebcc6f28f2d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:17.321354 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8ad25f-c336-4726-8086-11a80f16ac82 service cinder] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:17.321580 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8ad25f-c336-4726-8086-11a80f16ac82 service cinder] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:17.321992 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8ad25f-c336-4726-8086-11a80f16ac82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:17.332261 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8ad25f-c336-4726-8086-11a80f16ac82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:17.334149 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8ad25f-c336-4726-8086-11a80f16ac82 service cinder] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:17.339259 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-050426cf-5f37-4171-a00d-013c6b2543f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85eac120275644fab4c388555a03fa70', 'request_id': 'req-050426cf-5f37-4171-a00d-013c6b2543f0'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:17.344076 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1030/2038] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:20:17 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:20:17.434904 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-050426cf-5f37-4171-a00d-013c6b2543f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85eac120275644fab4c388555a03fa70', 'request_id': 'req-050426cf-5f37-4171-a00d-013c6b2543f0'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:17.441225 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-050426cf-5f37-4171-a00d-013c6b2543f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85eac120275644fab4c388555a03fa70', 'request_id': 'req-050426cf-5f37-4171-a00d-013c6b2543f0'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:17.452928 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-050426cf-5f37-4171-a00d-013c6b2543f0 admin admin] The token cache is being invalidate because project 85eac120275644fab4c388555a03fa70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81995) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:20:17.453220 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-050426cf-5f37-4171-a00d-013c6b2543f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:17.453909 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-050426cf-5f37-4171-a00d-013c6b2543f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:17.455546 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1009/2039] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:20:17 2022] DELETE /identity/v3/projects/85eac120275644fab4c388555a03fa70 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:20:17.459710 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8ad25f-c336-4726-8086-11a80f16ac82 service cinder] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:17.472555 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8ad25f-c336-4726-8086-11a80f16ac82 service cinder] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:17.560095 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31344a73-58e8-42d5-9a3c-946f66029f9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:17.566042 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31344a73-58e8-42d5-9a3c-946f66029f9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:17.566042 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31344a73-58e8-42d5-9a3c-946f66029f9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:17.566042 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31344a73-58e8-42d5-9a3c-946f66029f9f admin admin] PATH_INFO: `/v3/users/eaa083869df34ff1a153981df0ee1143` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:17.566042 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31344a73-58e8-42d5-9a3c-946f66029f9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=eaa083869df34ff1a153981df0ee1143)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:17.566042 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31344a73-58e8-42d5-9a3c-946f66029f9f admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:17.573620 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050426cf-5f37-4171-a00d-013c6b2543f0 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:17.575398 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-050426cf-5f37-4171-a00d-013c6b2543f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:17.635112 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050426cf-5f37-4171-a00d-013c6b2543f0 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:17.639055 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69189a5b-7252-4124-94e9-ea74cc0f7a3a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60193ba9e1c14df3bc9eebcc6f28f2d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:17.639669 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69189a5b-7252-4124-94e9-ea74cc0f7a3a service cinder] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:17.639951 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69189a5b-7252-4124-94e9-ea74cc0f7a3a service cinder] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:17.640227 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69189a5b-7252-4124-94e9-ea74cc0f7a3a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:17.641956 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-69189a5b-7252-4124-94e9-ea74cc0f7a3a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:17.667893 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:INSERT=3,keystone:SELECT=90,keystone:DELETE=7 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:20:17.701031 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69189a5b-7252-4124-94e9-ea74cc0f7a3a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:17.702469 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69189a5b-7252-4124-94e9-ea74cc0f7a3a service cinder] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:17.712137 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1010/2040] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:20:17 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 25 03:20:17.785058 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-31344a73-58e8-42d5-9a3c-946f66029f9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eaa083869df34ff1a153981df0ee1143'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:17.886545 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-31344a73-58e8-42d5-9a3c-946f66029f9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eaa083869df34ff1a153981df0ee1143'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:17.887164 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-31344a73-58e8-42d5-9a3c-946f66029f9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eaa083869df34ff1a153981df0ee1143'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:17.897007 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-31344a73-58e8-42d5-9a3c-946f66029f9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eaa083869df34ff1a153981df0ee1143'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:17.911911 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-31344a73-58e8-42d5-9a3c-946f66029f9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eaa083869df34ff1a153981df0ee1143'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:17.918453 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1031/2041] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:20:17 2022] DELETE /identity/v3/users/eaa083869df34ff1a153981df0ee1143 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:20:18.385527 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69189a5b-7252-4124-94e9-ea74cc0f7a3a service cinder] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:18.387031 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-69189a5b-7252-4124-94e9-ea74cc0f7a3a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:18.401379 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69189a5b-7252-4124-94e9-ea74cc0f7a3a service cinder] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:18.458341 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ea2603-8e05-4deb-bdfa-7103e14de964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:18.458985 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ea2603-8e05-4deb-bdfa-7103e14de964 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:18.459268 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ea2603-8e05-4deb-bdfa-7103e14de964 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:18.459547 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ea2603-8e05-4deb-bdfa-7103e14de964 admin admin] PATH_INFO: `/v3/projects/506a9ad77ba8400a889b0d0527f658fd` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:18.460694 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ea2603-8e05-4deb-bdfa-7103e14de964 admin admin] RBAC: Authorizing `identity:delete_project(project_id=506a9ad77ba8400a889b0d0527f658fd)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:18.462640 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ea2603-8e05-4deb-bdfa-7103e14de964 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:18.573628 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-16ea2603-8e05-4deb-bdfa-7103e14de964 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '506a9ad77ba8400a889b0d0527f658fd', 'request_id': 'req-16ea2603-8e05-4deb-bdfa-7103e14de964'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:18.646372 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-16ea2603-8e05-4deb-bdfa-7103e14de964 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '506a9ad77ba8400a889b0d0527f658fd', 'request_id': 'req-16ea2603-8e05-4deb-bdfa-7103e14de964'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:18.653070 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-16ea2603-8e05-4deb-bdfa-7103e14de964 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '506a9ad77ba8400a889b0d0527f658fd', 'request_id': 'req-16ea2603-8e05-4deb-bdfa-7103e14de964'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:18.666962 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-16ea2603-8e05-4deb-bdfa-7103e14de964 admin admin] The token cache is being invalidate because project 506a9ad77ba8400a889b0d0527f658fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81995) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:20:18.667400 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-16ea2603-8e05-4deb-bdfa-7103e14de964 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:18.668593 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-16ea2603-8e05-4deb-bdfa-7103e14de964 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:18.671227 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1011/2042] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:20:18 2022] DELETE /identity/v3/projects/506a9ad77ba8400a889b0d0527f658fd => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:20:18.677438 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8794097-3077-4662-be1f-17534a24ddce None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:18.677725 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8794097-3077-4662-be1f-17534a24ddce None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:18.678003 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8794097-3077-4662-be1f-17534a24ddce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:18.699468 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-f8794097-3077-4662-be1f-17534a24ddce None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:20:18.700713 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-f8794097-3077-4662-be1f-17534a24ddce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:18.706324 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:20:18.767608 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1032/2043] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:20:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:20:18.775565 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b48f123-8348-4e4f-aa85-3141db41e923 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:18.775930 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b48f123-8348-4e4f-aa85-3141db41e923 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:18.776176 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b48f123-8348-4e4f-aa85-3141db41e923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:18.796678 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-6b48f123-8348-4e4f-aa85-3141db41e923 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:20:18.797942 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-6b48f123-8348-4e4f-aa85-3141db41e923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:18.806841 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1012/2044] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:20:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:20:18.810782 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8794097-3077-4662-be1f-17534a24ddce None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:18.820490 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8794097-3077-4662-be1f-17534a24ddce None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:18.825502 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa2c0ce-812e-48aa-ba83-a88be20013b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:18.826105 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa2c0ce-812e-48aa-ba83-a88be20013b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:18.826420 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa2c0ce-812e-48aa-ba83-a88be20013b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:18.826699 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa2c0ce-812e-48aa-ba83-a88be20013b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:18.827462 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa2c0ce-812e-48aa-ba83-a88be20013b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:20:18.827980 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa2c0ce-812e-48aa-ba83-a88be20013b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:18.830221 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa2c0ce-812e-48aa-ba83-a88be20013b5 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:18.841862 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1033/2045] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Aug 25 03:20:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:20:18.847064 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b48f123-8348-4e4f-aa85-3141db41e923 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:18.856180 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b48f123-8348-4e4f-aa85-3141db41e923 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:18.860924 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a22843-ede5-472d-bffd-1211a171b8b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:18.862075 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a22843-ede5-472d-bffd-1211a171b8b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:18.862386 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a22843-ede5-472d-bffd-1211a171b8b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:18.862667 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a22843-ede5-472d-bffd-1211a171b8b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:18.863300 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a22843-ede5-472d-bffd-1211a171b8b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:18.865267 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a22843-ede5-472d-bffd-1211a171b8b7 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:18.983338 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1013/2046] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:20:18 2022] POST /identity/v3/projects => generated 385 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:20:18.984556 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa2c0ce-812e-48aa-ba83-a88be20013b5 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:18.995384 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa2c0ce-812e-48aa-ba83-a88be20013b5 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:19.056658 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de19800-d558-4de5-afe6-d41b82efe6dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:19.057352 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de19800-d558-4de5-afe6-d41b82efe6dc admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:19.057542 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de19800-d558-4de5-afe6-d41b82efe6dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:19.057815 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de19800-d558-4de5-afe6-d41b82efe6dc admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:19.058499 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de19800-d558-4de5-afe6-d41b82efe6dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:19.060573 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de19800-d558-4de5-afe6-d41b82efe6dc admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:19.138913 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1034/2047] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:20:18 2022] POST /identity/v3/users => generated 293 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:20:19.146079 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a22843-ede5-472d-bffd-1211a171b8b7 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:19.156479 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a22843-ede5-472d-bffd-1211a171b8b7 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:19.162621 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a525091d-e835-4df9-902c-bfc961a47bb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:19.163626 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a525091d-e835-4df9-902c-bfc961a47bb9 admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:19.163963 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a525091d-e835-4df9-902c-bfc961a47bb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:19.164241 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a525091d-e835-4df9-902c-bfc961a47bb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:19.165129 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a525091d-e835-4df9-902c-bfc961a47bb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:19.166988 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a525091d-e835-4df9-902c-bfc961a47bb9 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:19.179119 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1014/2048] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:20:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:20:19.182908 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de19800-d558-4de5-afe6-d41b82efe6dc admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:19.195722 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de19800-d558-4de5-afe6-d41b82efe6dc admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:19.200853 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34b9ddb-780c-45c8-93e0-7542736bfbe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:19.204100 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34b9ddb-780c-45c8-93e0-7542736bfbe5 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:19.204100 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34b9ddb-780c-45c8-93e0-7542736bfbe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:19.204100 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34b9ddb-780c-45c8-93e0-7542736bfbe5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:19.204100 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34b9ddb-780c-45c8-93e0-7542736bfbe5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:19.204452 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34b9ddb-780c-45c8-93e0-7542736bfbe5 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:19.215348 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1035/2049] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:20:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:20:19.219539 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a525091d-e835-4df9-902c-bfc961a47bb9 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:19.230247 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a525091d-e835-4df9-902c-bfc961a47bb9 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:19.234976 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df1a9d6-54e3-478b-8ddb-1a31806abdd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:19.235694 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df1a9d6-54e3-478b-8ddb-1a31806abdd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:19.236019 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df1a9d6-54e3-478b-8ddb-1a31806abdd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:19.236294 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df1a9d6-54e3-478b-8ddb-1a31806abdd5 admin admin] PATH_INFO: `/v3/projects/2dd29de1c5464ba2a63143bb690a4c9f/users/ddf0f2c4e6574edd91d6f666b8bb13e8/roles/3255f77192524a16bc405173106871fc` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:19.248979 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:20:19.252099 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df1a9d6-54e3-478b-8ddb-1a31806abdd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2dd29de1c5464ba2a63143bb690a4c9f, user_id=ddf0f2c4e6574edd91d6f666b8bb13e8, role_id=3255f77192524a16bc405173106871fc)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:19.255685 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df1a9d6-54e3-478b-8ddb-1a31806abdd5 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:19.362910 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1015/2050] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:20:19 2022] PUT /identity/v3/projects/2dd29de1c5464ba2a63143bb690a4c9f/users/ddf0f2c4e6574edd91d6f666b8bb13e8/roles/3255f77192524a16bc405173106871fc => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:20:19.374936 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96726cbb-7d13-48b9-8900-96aa21e69916 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:19.375230 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96726cbb-7d13-48b9-8900-96aa21e69916 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:19.378515 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96726cbb-7d13-48b9-8900-96aa21e69916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:19.407483 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-96726cbb-7d13-48b9-8900-96aa21e69916 None None] MFA Rules not processed for user `ddf0f2c4e6574edd91d6f666b8bb13e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:20:19.409372 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-96726cbb-7d13-48b9-8900-96aa21e69916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:19.482907 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1036/2051] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:20:19 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:20:19.494183 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df1a9d6-54e3-478b-8ddb-1a31806abdd5 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:19.495957 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-8df1a9d6-54e3-478b-8ddb-1a31806abdd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:19.558563 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df1a9d6-54e3-478b-8ddb-1a31806abdd5 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:19.562177 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3362226-ef19-4142-abfe-41d3865c058b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60193ba9e1c14df3bc9eebcc6f28f2d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:19.562811 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3362226-ef19-4142-abfe-41d3865c058b service cinder] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:19.563145 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3362226-ef19-4142-abfe-41d3865c058b service cinder] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:19.563422 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3362226-ef19-4142-abfe-41d3865c058b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:19.572540 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3362226-ef19-4142-abfe-41d3865c058b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:19.573937 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3362226-ef19-4142-abfe-41d3865c058b service cinder] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:19.584797 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1016/2052] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:20:19 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:20:26.208079 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96726cbb-7d13-48b9-8900-96aa21e69916 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:26.210214 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-96726cbb-7d13-48b9-8900-96aa21e69916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:26.216615 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:20:26.279159 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96726cbb-7d13-48b9-8900-96aa21e69916 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:26.286530 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1a5433-e1cf-40c9-953b-ba985809b601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:26.287794 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1a5433-e1cf-40c9-953b-ba985809b601 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:26.287988 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1a5433-e1cf-40c9-953b-ba985809b601 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:26.288312 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1a5433-e1cf-40c9-953b-ba985809b601 admin admin] PATH_INFO: `/v3/users/16ba7682df534212ace9509688c0c183` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:26.289726 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1a5433-e1cf-40c9-953b-ba985809b601 admin admin] RBAC: Authorizing `identity:delete_user(user_id=16ba7682df534212ace9509688c0c183)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:26.292479 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1a5433-e1cf-40c9-953b-ba985809b601 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:26.441965 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-2a1a5433-e1cf-40c9-953b-ba985809b601 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16ba7682df534212ace9509688c0c183'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:26.497972 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-2a1a5433-e1cf-40c9-953b-ba985809b601 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16ba7682df534212ace9509688c0c183'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:26.498657 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-2a1a5433-e1cf-40c9-953b-ba985809b601 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16ba7682df534212ace9509688c0c183'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:26.506642 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-2a1a5433-e1cf-40c9-953b-ba985809b601 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16ba7682df534212ace9509688c0c183'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:26.519169 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-2a1a5433-e1cf-40c9-953b-ba985809b601 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16ba7682df534212ace9509688c0c183'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:26.525609 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1037/2053] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:20:26 2022] DELETE /identity/v3/users/16ba7682df534212ace9509688c0c183 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:20:26.530048 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd150920-aca3-416d-bc4d-47e2e2ed757c None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:26.530576 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd150920-aca3-416d-bc4d-47e2e2ed757c None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:26.530681 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd150920-aca3-416d-bc4d-47e2e2ed757c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:26.550550 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-cd150920-aca3-416d-bc4d-47e2e2ed757c None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:20:26.551699 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-cd150920-aca3-416d-bc4d-47e2e2ed757c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:26.614898 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1017/2054] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:20:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:20:26.624893 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1a5433-e1cf-40c9-953b-ba985809b601 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:26.626522 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-2a1a5433-e1cf-40c9-953b-ba985809b601 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:26.685520 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1a5433-e1cf-40c9-953b-ba985809b601 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:26.690491 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c4ca2d-4146-4ebf-ae01-51ded2935d41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:26.690770 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4ca2d-4146-4ebf-ae01-51ded2935d41 service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:26.696886 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4ca2d-4146-4ebf-ae01-51ded2935d41 service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:26.696886 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4ca2d-4146-4ebf-ae01-51ded2935d41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:26.702775 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c4ca2d-4146-4ebf-ae01-51ded2935d41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:26.704226 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c4ca2d-4146-4ebf-ae01-51ded2935d41 service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:26.720043 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1038/2055] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:20:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:20:27.262655 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd150920-aca3-416d-bc4d-47e2e2ed757c None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:27.264858 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-cd150920-aca3-416d-bc4d-47e2e2ed757c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:27.283057 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd150920-aca3-416d-bc4d-47e2e2ed757c None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:27.288569 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdb61b4-8a02-4295-8051-e02d5f44ab7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:27.289412 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdb61b4-8a02-4295-8051-e02d5f44ab7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:27.289819 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdb61b4-8a02-4295-8051-e02d5f44ab7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:27.290163 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdb61b4-8a02-4295-8051-e02d5f44ab7e admin admin] PATH_INFO: `/v3/projects/fe407cc5a52940c585bb1d2d9efd7c07` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:27.291814 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdb61b4-8a02-4295-8051-e02d5f44ab7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe407cc5a52940c585bb1d2d9efd7c07)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:27.294693 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdb61b4-8a02-4295-8051-e02d5f44ab7e admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:27.441856 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-8fdb61b4-8a02-4295-8051-e02d5f44ab7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe407cc5a52940c585bb1d2d9efd7c07', 'request_id': 'req-8fdb61b4-8a02-4295-8051-e02d5f44ab7e'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:27.537356 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-8fdb61b4-8a02-4295-8051-e02d5f44ab7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe407cc5a52940c585bb1d2d9efd7c07', 'request_id': 'req-8fdb61b4-8a02-4295-8051-e02d5f44ab7e'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:27.543541 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-8fdb61b4-8a02-4295-8051-e02d5f44ab7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe407cc5a52940c585bb1d2d9efd7c07', 'request_id': 'req-8fdb61b4-8a02-4295-8051-e02d5f44ab7e'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:27.555212 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-8fdb61b4-8a02-4295-8051-e02d5f44ab7e admin admin] The token cache is being invalidate because project fe407cc5a52940c585bb1d2d9efd7c07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81995) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:20:27.555503 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-8fdb61b4-8a02-4295-8051-e02d5f44ab7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:27.556145 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-8fdb61b4-8a02-4295-8051-e02d5f44ab7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:27.557638 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1018/2056] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:20:27 2022] DELETE /identity/v3/projects/fe407cc5a52940c585bb1d2d9efd7c07 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:20:27.561612 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c4ca2d-4146-4ebf-ae01-51ded2935d41 service neutron] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:27.572290 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c4ca2d-4146-4ebf-ae01-51ded2935d41 service neutron] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:27.597353 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:20:27.630398 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e2af1a-66b9-450a-91f8-68b705109373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:27.631030 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e2af1a-66b9-450a-91f8-68b705109373 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:27.631314 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e2af1a-66b9-450a-91f8-68b705109373 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:27.631586 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e2af1a-66b9-450a-91f8-68b705109373 admin admin] PATH_INFO: `/v3/users/1e3d14da2c5a4e4886f7d28dcc0913f3` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:27.632824 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e2af1a-66b9-450a-91f8-68b705109373 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e3d14da2c5a4e4886f7d28dcc0913f3)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:27.634898 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e2af1a-66b9-450a-91f8-68b705109373 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:27.829522 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-c6e2af1a-66b9-450a-91f8-68b705109373 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e3d14da2c5a4e4886f7d28dcc0913f3'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:27.938240 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-c6e2af1a-66b9-450a-91f8-68b705109373 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e3d14da2c5a4e4886f7d28dcc0913f3'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:27.938979 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-c6e2af1a-66b9-450a-91f8-68b705109373 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e3d14da2c5a4e4886f7d28dcc0913f3'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:27.951180 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-c6e2af1a-66b9-450a-91f8-68b705109373 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e3d14da2c5a4e4886f7d28dcc0913f3'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:27.976652 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-c6e2af1a-66b9-450a-91f8-68b705109373 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e3d14da2c5a4e4886f7d28dcc0913f3'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:27.986248 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1039/2057] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:20:27 2022] DELETE /identity/v3/users/1e3d14da2c5a4e4886f7d28dcc0913f3 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:20:28.436455 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdb61b4-8a02-4295-8051-e02d5f44ab7e admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:28.437967 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-8fdb61b4-8a02-4295-8051-e02d5f44ab7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:28.453433 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdb61b4-8a02-4295-8051-e02d5f44ab7e admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:28.467923 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:20:28.513109 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469ead9d-94bc-47b8-b7d1-d4845d690065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:28.514131 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ead9d-94bc-47b8-b7d1-d4845d690065 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:28.514455 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ead9d-94bc-47b8-b7d1-d4845d690065 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:28.514735 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ead9d-94bc-47b8-b7d1-d4845d690065 admin admin] PATH_INFO: `/v3/projects/2b56f28119b749acaae9738160cd5406` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:28.515932 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469ead9d-94bc-47b8-b7d1-d4845d690065 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b56f28119b749acaae9738160cd5406)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:28.517863 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469ead9d-94bc-47b8-b7d1-d4845d690065 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:28.614266 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-469ead9d-94bc-47b8-b7d1-d4845d690065 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b56f28119b749acaae9738160cd5406', 'request_id': 'req-469ead9d-94bc-47b8-b7d1-d4845d690065'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:28.692450 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-469ead9d-94bc-47b8-b7d1-d4845d690065 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b56f28119b749acaae9738160cd5406', 'request_id': 'req-469ead9d-94bc-47b8-b7d1-d4845d690065'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:28.698585 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-469ead9d-94bc-47b8-b7d1-d4845d690065 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b56f28119b749acaae9738160cd5406', 'request_id': 'req-469ead9d-94bc-47b8-b7d1-d4845d690065'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:28.710826 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-469ead9d-94bc-47b8-b7d1-d4845d690065 admin admin] The token cache is being invalidate because project 2b56f28119b749acaae9738160cd5406 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81995) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:20:28.711118 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-469ead9d-94bc-47b8-b7d1-d4845d690065 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:28.711775 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-469ead9d-94bc-47b8-b7d1-d4845d690065 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:28.713362 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1019/2058] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:20:28 2022] DELETE /identity/v3/projects/2b56f28119b749acaae9738160cd5406 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:20:28.718221 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb3a485-3f51-4096-993b-57832579c9d5 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:28.722693 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb3a485-3f51-4096-993b-57832579c9d5 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:28.722693 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb3a485-3f51-4096-993b-57832579c9d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:28.738051 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-eeb3a485-3f51-4096-993b-57832579c9d5 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:20:28.739284 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-eeb3a485-3f51-4096-993b-57832579c9d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:28.798523 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1040/2059] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:20:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:20:28.806473 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b9396c-2d0f-4baf-b47a-72fde3ac43d8 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:28.806766 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b9396c-2d0f-4baf-b47a-72fde3ac43d8 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:28.807065 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b9396c-2d0f-4baf-b47a-72fde3ac43d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:28.827435 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-e1b9396c-2d0f-4baf-b47a-72fde3ac43d8 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:20:28.828638 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-e1b9396c-2d0f-4baf-b47a-72fde3ac43d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:28.837282 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1020/2060] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:20:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:20:28.841516 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb3a485-3f51-4096-993b-57832579c9d5 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:28.851022 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb3a485-3f51-4096-993b-57832579c9d5 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:28.855551 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feb8d11-c260-4d51-b198-6829eb215b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:28.856203 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feb8d11-c260-4d51-b198-6829eb215b1a admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:28.856470 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feb8d11-c260-4d51-b198-6829eb215b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:28.856747 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feb8d11-c260-4d51-b198-6829eb215b1a admin admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:28.857392 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2feb8d11-c260-4d51-b198-6829eb215b1a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:20:28.857695 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2feb8d11-c260-4d51-b198-6829eb215b1a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:28.859324 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2feb8d11-c260-4d51-b198-6829eb215b1a admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:28.871169 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1041/2061] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Aug 25 03:20:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:20:28.875260 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b9396c-2d0f-4baf-b47a-72fde3ac43d8 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:28.885295 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b9396c-2d0f-4baf-b47a-72fde3ac43d8 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:28.889944 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e0beb3-9fbf-491b-bd01-4cac43eb45df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:28.890562 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e0beb3-9fbf-491b-bd01-4cac43eb45df admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:28.890840 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e0beb3-9fbf-491b-bd01-4cac43eb45df admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:28.891113 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e0beb3-9fbf-491b-bd01-4cac43eb45df admin admin] PATH_INFO: `/v3/projects` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:28.891770 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e0beb3-9fbf-491b-bd01-4cac43eb45df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:28.893763 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e0beb3-9fbf-491b-bd01-4cac43eb45df admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:28.972171 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1021/2062] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:20:28 2022] POST /identity/v3/projects => generated 393 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:20:28.976440 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feb8d11-c260-4d51-b198-6829eb215b1a admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:28.986574 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feb8d11-c260-4d51-b198-6829eb215b1a admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:29.017518 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:20:29.044394 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76146457-da60-4f1f-be09-41bad3d16209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:29.045009 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76146457-da60-4f1f-be09-41bad3d16209 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:29.045291 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76146457-da60-4f1f-be09-41bad3d16209 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:29.045566 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76146457-da60-4f1f-be09-41bad3d16209 admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:29.046187 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76146457-da60-4f1f-be09-41bad3d16209 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:29.048207 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76146457-da60-4f1f-be09-41bad3d16209 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:29.155986 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1042/2063] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:20:28 2022] POST /identity/v3/users => generated 297 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:20:29.159931 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e0beb3-9fbf-491b-bd01-4cac43eb45df admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:29.170978 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e0beb3-9fbf-491b-bd01-4cac43eb45df admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:29.176590 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510910af-7311-4ce4-8eb1-b26d399df894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:29.177204 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510910af-7311-4ce4-8eb1-b26d399df894 admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:29.177485 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510910af-7311-4ce4-8eb1-b26d399df894 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:29.177759 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510910af-7311-4ce4-8eb1-b26d399df894 admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:29.178447 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510910af-7311-4ce4-8eb1-b26d399df894 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:29.180087 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510910af-7311-4ce4-8eb1-b26d399df894 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:29.191849 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1022/2064] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:20:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:20:29.195843 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76146457-da60-4f1f-be09-41bad3d16209 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:29.206868 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76146457-da60-4f1f-be09-41bad3d16209 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:29.212184 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2b1479-b6ff-48d8-9391-880ae88b4d4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:29.213031 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2b1479-b6ff-48d8-9391-880ae88b4d4f admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:29.213662 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2b1479-b6ff-48d8-9391-880ae88b4d4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:29.214024 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2b1479-b6ff-48d8-9391-880ae88b4d4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:29.214810 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2b1479-b6ff-48d8-9391-880ae88b4d4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:29.216460 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2b1479-b6ff-48d8-9391-880ae88b4d4f admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:29.226953 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1043/2065] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:20:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:20:29.231233 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510910af-7311-4ce4-8eb1-b26d399df894 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:29.242048 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510910af-7311-4ce4-8eb1-b26d399df894 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:29.246833 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304efcfe-94a3-4ad9-b7f5-a4f0258636f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:29.247491 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304efcfe-94a3-4ad9-b7f5-a4f0258636f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:29.247774 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304efcfe-94a3-4ad9-b7f5-a4f0258636f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:29.248053 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304efcfe-94a3-4ad9-b7f5-a4f0258636f8 admin admin] PATH_INFO: `/v3/projects/7cbef142fa684e63bc8e786b138dd953/users/ed7c14e5c39342129a1a2e5502a1524d/roles/3255f77192524a16bc405173106871fc` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:29.263889 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304efcfe-94a3-4ad9-b7f5-a4f0258636f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cbef142fa684e63bc8e786b138dd953, user_id=ed7c14e5c39342129a1a2e5502a1524d, role_id=3255f77192524a16bc405173106871fc)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:29.266584 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304efcfe-94a3-4ad9-b7f5-a4f0258636f8 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:29.349944 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1023/2066] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:20:29 2022] PUT /identity/v3/projects/7cbef142fa684e63bc8e786b138dd953/users/ed7c14e5c39342129a1a2e5502a1524d/roles/3255f77192524a16bc405173106871fc => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:20:29.360424 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89454ae2-ebb5-45f9-b1f4-fbeacb12a454 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:29.360714 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89454ae2-ebb5-45f9-b1f4-fbeacb12a454 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:29.360987 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89454ae2-ebb5-45f9-b1f4-fbeacb12a454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:29.390483 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-89454ae2-ebb5-45f9-b1f4-fbeacb12a454 None None] MFA Rules not processed for user `ed7c14e5c39342129a1a2e5502a1524d`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:20:29.391683 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-89454ae2-ebb5-45f9-b1f4-fbeacb12a454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:29.463956 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1044/2067] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:20:29 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:20:29.468429 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304efcfe-94a3-4ad9-b7f5-a4f0258636f8 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:29.515744 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:20:29.531607 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304efcfe-94a3-4ad9-b7f5-a4f0258636f8 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:29.536842 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66e0fb2-da95-4498-9b17-2ed126f55857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:29.537459 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66e0fb2-da95-4498-9b17-2ed126f55857 admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:29.537743 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66e0fb2-da95-4498-9b17-2ed126f55857 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:29.538017 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66e0fb2-da95-4498-9b17-2ed126f55857 admin admin] PATH_INFO: `/v3/projects` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:29.538687 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66e0fb2-da95-4498-9b17-2ed126f55857 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:29.540590 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66e0fb2-da95-4498-9b17-2ed126f55857 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:29.685660 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1024/2068] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:20:29 2022] POST /identity/v3/projects => generated 391 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:20:29.686716 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89454ae2-ebb5-45f9-b1f4-fbeacb12a454 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:29.698354 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89454ae2-ebb5-45f9-b1f4-fbeacb12a454 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:29.759547 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d660fb60-b652-4b5d-8c4b-3c86df54477d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:29.767026 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d660fb60-b652-4b5d-8c4b-3c86df54477d admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:29.767026 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d660fb60-b652-4b5d-8c4b-3c86df54477d admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:29.767026 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d660fb60-b652-4b5d-8c4b-3c86df54477d admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:29.767026 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d660fb60-b652-4b5d-8c4b-3c86df54477d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:29.767026 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d660fb60-b652-4b5d-8c4b-3c86df54477d admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:29.838132 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1045/2069] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:20:29 2022] POST /identity/v3/users => generated 302 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:20:29.842366 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66e0fb2-da95-4498-9b17-2ed126f55857 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:29.850625 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66e0fb2-da95-4498-9b17-2ed126f55857 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:29.855321 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7895c1aa-ada8-4318-8fe1-ad54bd080eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:29.855940 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7895c1aa-ada8-4318-8fe1-ad54bd080eb2 admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:29.856220 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7895c1aa-ada8-4318-8fe1-ad54bd080eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:29.856500 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7895c1aa-ada8-4318-8fe1-ad54bd080eb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:29.857154 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7895c1aa-ada8-4318-8fe1-ad54bd080eb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:29.858794 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7895c1aa-ada8-4318-8fe1-ad54bd080eb2 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:29.869315 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1025/2070] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:20:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:20:29.873745 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d660fb60-b652-4b5d-8c4b-3c86df54477d admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:29.884277 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d660fb60-b652-4b5d-8c4b-3c86df54477d admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:29.888953 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640d0500-db37-459e-a71e-62f34d276b41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:29.889739 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640d0500-db37-459e-a71e-62f34d276b41 admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:29.889916 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640d0500-db37-459e-a71e-62f34d276b41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:29.890199 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640d0500-db37-459e-a71e-62f34d276b41 admin admin] PATH_INFO: `/v3/projects/66cb8c8f830b40209e29cf866b250b63/users/f36bf528908a453991bfd0c17a107393/roles/62f19ea272b447ccaab1576cd9bf4fd9` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:29.906482 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640d0500-db37-459e-a71e-62f34d276b41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66cb8c8f830b40209e29cf866b250b63, user_id=f36bf528908a453991bfd0c17a107393, role_id=62f19ea272b447ccaab1576cd9bf4fd9)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:29.909181 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640d0500-db37-459e-a71e-62f34d276b41 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:29.988907 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1046/2071] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:20:29 2022] PUT /identity/v3/projects/66cb8c8f830b40209e29cf866b250b63/users/f36bf528908a453991bfd0c17a107393/roles/62f19ea272b447ccaab1576cd9bf4fd9 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:20:29.992865 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7895c1aa-ada8-4318-8fe1-ad54bd080eb2 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:30.002979 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7895c1aa-ada8-4318-8fe1-ad54bd080eb2 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:30.069017 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76563210-71ff-4e0f-80c1-ad8950d7605e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:30.069837 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76563210-71ff-4e0f-80c1-ad8950d7605e admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:30.070190 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76563210-71ff-4e0f-80c1-ad8950d7605e admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:30.070570 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76563210-71ff-4e0f-80c1-ad8950d7605e admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:30.071359 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76563210-71ff-4e0f-80c1-ad8950d7605e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:30.073228 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76563210-71ff-4e0f-80c1-ad8950d7605e admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:30.085841 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1026/2072] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:20:29 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:20:30.089990 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640d0500-db37-459e-a71e-62f34d276b41 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:30.102485 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640d0500-db37-459e-a71e-62f34d276b41 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:30.108550 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8a4f40-9dc3-440a-87f7-428f81734b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:30.109489 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8a4f40-9dc3-440a-87f7-428f81734b6e admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:30.109868 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8a4f40-9dc3-440a-87f7-428f81734b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:30.110418 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8a4f40-9dc3-440a-87f7-428f81734b6e admin admin] PATH_INFO: `/v3/projects/66cb8c8f830b40209e29cf866b250b63/users/f36bf528908a453991bfd0c17a107393/roles/62f19ea272b447ccaab1576cd9bf4fd9` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:30.112502 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8a4f40-9dc3-440a-87f7-428f81734b6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=66cb8c8f830b40209e29cf866b250b63, user_id=f36bf528908a453991bfd0c17a107393, role_id=62f19ea272b447ccaab1576cd9bf4fd9)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:30.115298 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8a4f40-9dc3-440a-87f7-428f81734b6e admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:30.118060 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:20:30.128887 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1047/2073] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:20:30 2022] PUT /identity/v3/projects/66cb8c8f830b40209e29cf866b250b63/users/f36bf528908a453991bfd0c17a107393/roles/62f19ea272b447ccaab1576cd9bf4fd9 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:20:30.151720 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ffac33-cebd-4ce5-93ac-9dae465f2f63 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:30.152009 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ffac33-cebd-4ce5-93ac-9dae465f2f63 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:30.152289 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ffac33-cebd-4ce5-93ac-9dae465f2f63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:30.181313 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-94ffac33-cebd-4ce5-93ac-9dae465f2f63 None None] MFA Rules not processed for user `f36bf528908a453991bfd0c17a107393`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:20:30.182565 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-94ffac33-cebd-4ce5-93ac-9dae465f2f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:30.265029 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1027/2074] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:20:30 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:20:30.272952 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8a4f40-9dc3-440a-87f7-428f81734b6e admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:30.274812 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-0f8a4f40-9dc3-440a-87f7-428f81734b6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:30.338667 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8a4f40-9dc3-440a-87f7-428f81734b6e admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:30.342309 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b1ef4e-a488-471b-b55c-6aed003bb3e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60193ba9e1c14df3bc9eebcc6f28f2d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:30.342925 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b1ef4e-a488-471b-b55c-6aed003bb3e0 service cinder] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:30.343198 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b1ef4e-a488-471b-b55c-6aed003bb3e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:30.343468 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b1ef4e-a488-471b-b55c-6aed003bb3e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:30.352999 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b1ef4e-a488-471b-b55c-6aed003bb3e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:30.354455 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b1ef4e-a488-471b-b55c-6aed003bb3e0 service cinder] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:30.364398 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1048/2075] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:20:30 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:20:30.415745 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ffac33-cebd-4ce5-93ac-9dae465f2f63 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:30.417379 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-94ffac33-cebd-4ce5-93ac-9dae465f2f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:30.429913 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ffac33-cebd-4ce5-93ac-9dae465f2f63 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:30.433688 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5190766a-c131-401f-8b6e-4f8e708f174c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60193ba9e1c14df3bc9eebcc6f28f2d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:30.434348 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5190766a-c131-401f-8b6e-4f8e708f174c service cinder] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:30.450899 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5190766a-c131-401f-8b6e-4f8e708f174c service cinder] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:30.450899 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5190766a-c131-401f-8b6e-4f8e708f174c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:30.479768 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:20:30.498650 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5190766a-c131-401f-8b6e-4f8e708f174c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:30.500088 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5190766a-c131-401f-8b6e-4f8e708f174c service cinder] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:30.509840 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1028/2076] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:20:30 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:20:35.007700 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b1ef4e-a488-471b-b55c-6aed003bb3e0 service cinder] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:35.016315 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b1ef4e-a488-471b-b55c-6aed003bb3e0 service cinder] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:35.020783 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f456cf-84b9-4250-b2a8-4b395fc0a713 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60193ba9e1c14df3bc9eebcc6f28f2d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:35.021461 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f456cf-84b9-4250-b2a8-4b395fc0a713 service cinder] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:35.021807 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f456cf-84b9-4250-b2a8-4b395fc0a713 service cinder] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:35.022145 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f456cf-84b9-4250-b2a8-4b395fc0a713 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:35.023968 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-22f456cf-84b9-4250-b2a8-4b395fc0a713 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:35.087908 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f456cf-84b9-4250-b2a8-4b395fc0a713 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:35.089339 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f456cf-84b9-4250-b2a8-4b395fc0a713 service cinder] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:35.098846 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1049/2077] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:20:35 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:20:38.498684 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5190766a-c131-401f-8b6e-4f8e708f174c service cinder] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:38.500257 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-5190766a-c131-401f-8b6e-4f8e708f174c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:38.585896 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5190766a-c131-401f-8b6e-4f8e708f174c service cinder] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:38.590620 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8785d2aa-919c-4395-a770-2d13651040be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:38.591268 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8785d2aa-919c-4395-a770-2d13651040be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:38.591566 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8785d2aa-919c-4395-a770-2d13651040be admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:38.591825 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8785d2aa-919c-4395-a770-2d13651040be admin admin] PATH_INFO: `/v3/users/ddf0f2c4e6574edd91d6f666b8bb13e8` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:38.592956 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8785d2aa-919c-4395-a770-2d13651040be admin admin] RBAC: Authorizing `identity:delete_user(user_id=ddf0f2c4e6574edd91d6f666b8bb13e8)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:38.595109 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8785d2aa-919c-4395-a770-2d13651040be admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:38.786941 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-8785d2aa-919c-4395-a770-2d13651040be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ddf0f2c4e6574edd91d6f666b8bb13e8'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:38.852009 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-8785d2aa-919c-4395-a770-2d13651040be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ddf0f2c4e6574edd91d6f666b8bb13e8'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:38.852629 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-8785d2aa-919c-4395-a770-2d13651040be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ddf0f2c4e6574edd91d6f666b8bb13e8'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:38.853238 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-8785d2aa-919c-4395-a770-2d13651040be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ddf0f2c4e6574edd91d6f666b8bb13e8'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:38.866821 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-8785d2aa-919c-4395-a770-2d13651040be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ddf0f2c4e6574edd91d6f666b8bb13e8'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:38.881040 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1029/2078] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:20:38 2022] DELETE /identity/v3/users/ddf0f2c4e6574edd91d6f666b8bb13e8 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:20:38.885786 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740f8563-4d6b-4fbd-aa89-0a4d3b6a17fb None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:38.886306 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740f8563-4d6b-4fbd-aa89-0a4d3b6a17fb None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:38.886398 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740f8563-4d6b-4fbd-aa89-0a4d3b6a17fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:38.911386 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-740f8563-4d6b-4fbd-aa89-0a4d3b6a17fb None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:20:38.911386 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-740f8563-4d6b-4fbd-aa89-0a4d3b6a17fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:38.912548 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8785d2aa-919c-4395-a770-2d13651040be admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:38.914532 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-8785d2aa-919c-4395-a770-2d13651040be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:38.971997 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1050/2079] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:20:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:20:38.986541 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740f8563-4d6b-4fbd-aa89-0a4d3b6a17fb None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:38.987916 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-740f8563-4d6b-4fbd-aa89-0a4d3b6a17fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:38.993131 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:20:38.997984 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8785d2aa-919c-4395-a770-2d13651040be admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:39.004369 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6b044a-75b1-4dcc-8ff5-3e6d72017515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:39.005075 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6b044a-75b1-4dcc-8ff5-3e6d72017515 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:39.005368 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6b044a-75b1-4dcc-8ff5-3e6d72017515 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:39.008192 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6b044a-75b1-4dcc-8ff5-3e6d72017515 admin admin] PATH_INFO: `/v3/users/ed7c14e5c39342129a1a2e5502a1524d` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:39.008192 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6b044a-75b1-4dcc-8ff5-3e6d72017515 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed7c14e5c39342129a1a2e5502a1524d)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:39.009023 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6b044a-75b1-4dcc-8ff5-3e6d72017515 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:39.026119 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:20:39.049328 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740f8563-4d6b-4fbd-aa89-0a4d3b6a17fb None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:39.052995 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da5d733-0e59-4c61-aec8-f40ad5f8a9ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:39.053608 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da5d733-0e59-4c61-aec8-f40ad5f8a9ee service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:39.053911 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da5d733-0e59-4c61-aec8-f40ad5f8a9ee service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:39.054187 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da5d733-0e59-4c61-aec8-f40ad5f8a9ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:39.063583 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da5d733-0e59-4c61-aec8-f40ad5f8a9ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:39.065008 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da5d733-0e59-4c61-aec8-f40ad5f8a9ee service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:39.075074 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1051/2080] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:20:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 25 03:20:39.216189 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-0e6b044a-75b1-4dcc-8ff5-3e6d72017515 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed7c14e5c39342129a1a2e5502a1524d'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:39.269213 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-0e6b044a-75b1-4dcc-8ff5-3e6d72017515 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed7c14e5c39342129a1a2e5502a1524d'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:39.270601 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-0e6b044a-75b1-4dcc-8ff5-3e6d72017515 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed7c14e5c39342129a1a2e5502a1524d'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:39.271225 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-0e6b044a-75b1-4dcc-8ff5-3e6d72017515 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed7c14e5c39342129a1a2e5502a1524d'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:39.294986 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-0e6b044a-75b1-4dcc-8ff5-3e6d72017515 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed7c14e5c39342129a1a2e5502a1524d'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:39.309287 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1030/2081] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:20:38 2022] DELETE /identity/v3/users/ed7c14e5c39342129a1a2e5502a1524d => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:20:39.313903 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f546985-74a3-47d5-ab82-3d89d4fcdf3f None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:39.314195 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f546985-74a3-47d5-ab82-3d89d4fcdf3f None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:39.314499 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f546985-74a3-47d5-ab82-3d89d4fcdf3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:39.334962 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-0f546985-74a3-47d5-ab82-3d89d4fcdf3f None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:20:39.336188 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-0f546985-74a3-47d5-ab82-3d89d4fcdf3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:39.404953 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1052/2082] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:20:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:20:39.416087 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6b044a-75b1-4dcc-8ff5-3e6d72017515 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:39.417529 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-0e6b044a-75b1-4dcc-8ff5-3e6d72017515 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:39.485027 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6b044a-75b1-4dcc-8ff5-3e6d72017515 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:39.492073 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac73bfe-48db-4a2f-b24f-11190285f246 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:39.494540 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f546985-74a3-47d5-ab82-3d89d4fcdf3f None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:39.496710 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-0f546985-74a3-47d5-ab82-3d89d4fcdf3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:39.497115 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac73bfe-48db-4a2f-b24f-11190285f246 service neutron] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:39.497565 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac73bfe-48db-4a2f-b24f-11190285f246 service neutron] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:39.498017 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac73bfe-48db-4a2f-b24f-11190285f246 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:39.523066 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f546985-74a3-47d5-ab82-3d89d4fcdf3f None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:39.527444 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac73bfe-48db-4a2f-b24f-11190285f246 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:39.530258 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac73bfe-48db-4a2f-b24f-11190285f246 service neutron] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:39.542806 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4917666e-ed8f-4c87-9a87-f9ad2c86275d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:39.543601 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4917666e-ed8f-4c87-9a87-f9ad2c86275d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:39.543858 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4917666e-ed8f-4c87-9a87-f9ad2c86275d admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:39.544134 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4917666e-ed8f-4c87-9a87-f9ad2c86275d admin admin] PATH_INFO: `/v3/projects/2dd29de1c5464ba2a63143bb690a4c9f` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:39.545336 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4917666e-ed8f-4c87-9a87-f9ad2c86275d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2dd29de1c5464ba2a63143bb690a4c9f)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:39.547322 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4917666e-ed8f-4c87-9a87-f9ad2c86275d admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:39.549004 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1031/2083] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:20:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:20:39.688879 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-4917666e-ed8f-4c87-9a87-f9ad2c86275d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2dd29de1c5464ba2a63143bb690a4c9f', 'request_id': 'req-4917666e-ed8f-4c87-9a87-f9ad2c86275d'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:39.801425 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-4917666e-ed8f-4c87-9a87-f9ad2c86275d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2dd29de1c5464ba2a63143bb690a4c9f', 'request_id': 'req-4917666e-ed8f-4c87-9a87-f9ad2c86275d'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:39.808068 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-4917666e-ed8f-4c87-9a87-f9ad2c86275d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2dd29de1c5464ba2a63143bb690a4c9f', 'request_id': 'req-4917666e-ed8f-4c87-9a87-f9ad2c86275d'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:39.823317 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-4917666e-ed8f-4c87-9a87-f9ad2c86275d admin admin] The token cache is being invalidate because project 2dd29de1c5464ba2a63143bb690a4c9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:20:39.823317 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-4917666e-ed8f-4c87-9a87-f9ad2c86275d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:39.825630 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-4917666e-ed8f-4c87-9a87-f9ad2c86275d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:39.830360 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1053/2084] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:20:39 2022] DELETE /identity/v3/projects/2dd29de1c5464ba2a63143bb690a4c9f => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:20:39.839040 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21e1755-b099-4511-af95-a236bf919228 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:39.839488 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21e1755-b099-4511-af95-a236bf919228 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:39.840074 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21e1755-b099-4511-af95-a236bf919228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:39.870204 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-c21e1755-b099-4511-af95-a236bf919228 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:20:39.872274 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-c21e1755-b099-4511-af95-a236bf919228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:39.952383 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1032/2085] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:20:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:20:39.960940 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e47c77-ec20-46ba-8ab9-dae2e51a4511 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:39.961741 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e47c77-ec20-46ba-8ab9-dae2e51a4511 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:39.961741 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e47c77-ec20-46ba-8ab9-dae2e51a4511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:39.986772 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-b4e47c77-ec20-46ba-8ab9-dae2e51a4511 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:20:39.989197 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-b4e47c77-ec20-46ba-8ab9-dae2e51a4511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:40.013502 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1054/2086] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:20:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:20:40.018782 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21e1755-b099-4511-af95-a236bf919228 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:40.030724 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21e1755-b099-4511-af95-a236bf919228 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:40.045535 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632211c1-47bf-44e4-b534-b2f54181bed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:40.046176 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632211c1-47bf-44e4-b534-b2f54181bed1 admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:40.046508 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632211c1-47bf-44e4-b534-b2f54181bed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:40.046789 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632211c1-47bf-44e4-b534-b2f54181bed1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:40.047440 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632211c1-47bf-44e4-b534-b2f54181bed1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:20:40.047745 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632211c1-47bf-44e4-b534-b2f54181bed1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:40.049588 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632211c1-47bf-44e4-b534-b2f54181bed1 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:40.066875 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1033/2087] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Aug 25 03:20:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:20:40.071917 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e47c77-ec20-46ba-8ab9-dae2e51a4511 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:40.083609 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e47c77-ec20-46ba-8ab9-dae2e51a4511 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:40.092446 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b3dec4-8074-4960-8baf-eb8e5a5e4fe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:40.093055 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b3dec4-8074-4960-8baf-eb8e5a5e4fe4 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:40.093344 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b3dec4-8074-4960-8baf-eb8e5a5e4fe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:40.093620 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b3dec4-8074-4960-8baf-eb8e5a5e4fe4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:40.094372 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b3dec4-8074-4960-8baf-eb8e5a5e4fe4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:40.096505 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b3dec4-8074-4960-8baf-eb8e5a5e4fe4 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:40.190111 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1055/2088] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:20:40 2022] POST /identity/v3/projects => generated 367 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:20:40.194544 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632211c1-47bf-44e4-b534-b2f54181bed1 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:40.205011 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632211c1-47bf-44e4-b534-b2f54181bed1 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:40.209242 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:20:40.244883 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b3dec4-8074-4960-8baf-eb8e5a5e4fe4 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:40.247333 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-04b3dec4-8074-4960-8baf-eb8e5a5e4fe4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:40.302726 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:20:40.308425 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a8b086-d4ea-4010-b24a-16e9bbd7f061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:40.309660 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a8b086-d4ea-4010-b24a-16e9bbd7f061 admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:40.310091 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a8b086-d4ea-4010-b24a-16e9bbd7f061 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:40.310545 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a8b086-d4ea-4010-b24a-16e9bbd7f061 admin admin] PATH_INFO: `/v3/users` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:40.311519 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a8b086-d4ea-4010-b24a-16e9bbd7f061 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:40.314906 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a8b086-d4ea-4010-b24a-16e9bbd7f061 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:40.340599 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b3dec4-8074-4960-8baf-eb8e5a5e4fe4 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:40.345457 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51beca4d-c152-48e2-aa55-35725fb9e9e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:40.346089 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51beca4d-c152-48e2-aa55-35725fb9e9e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:40.346405 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51beca4d-c152-48e2-aa55-35725fb9e9e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:40.346682 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51beca4d-c152-48e2-aa55-35725fb9e9e0 admin admin] PATH_INFO: `/v3/projects/7cbef142fa684e63bc8e786b138dd953` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:40.348443 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51beca4d-c152-48e2-aa55-35725fb9e9e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7cbef142fa684e63bc8e786b138dd953)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:40.351901 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51beca4d-c152-48e2-aa55-35725fb9e9e0 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:40.429547 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1034/2089] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:20:40 2022] POST /identity/v3/users => generated 284 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:20:40.433609 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a8b086-d4ea-4010-b24a-16e9bbd7f061 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:40.443155 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a8b086-d4ea-4010-b24a-16e9bbd7f061 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:40.447778 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccfe8fc-1de4-4f67-b048-9187d7dc5456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:40.448355 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccfe8fc-1de4-4f67-b048-9187d7dc5456 admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:40.448636 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccfe8fc-1de4-4f67-b048-9187d7dc5456 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:40.448910 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccfe8fc-1de4-4f67-b048-9187d7dc5456 admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:40.449840 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccfe8fc-1de4-4f67-b048-9187d7dc5456 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:40.451462 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccfe8fc-1de4-4f67-b048-9187d7dc5456 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:40.462929 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1035/2090] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:20:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 25 03:20:40.467937 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccfe8fc-1de4-4f67-b048-9187d7dc5456 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:40.478886 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-51beca4d-c152-48e2-aa55-35725fb9e9e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7cbef142fa684e63bc8e786b138dd953', 'request_id': 'req-51beca4d-c152-48e2-aa55-35725fb9e9e0'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:40.482415 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccfe8fc-1de4-4f67-b048-9187d7dc5456 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:40.554440 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a20a2f-76a7-414e-b491-17a5c2f8fe84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:40.555095 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a20a2f-76a7-414e-b491-17a5c2f8fe84 admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:40.555323 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a20a2f-76a7-414e-b491-17a5c2f8fe84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:40.555600 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a20a2f-76a7-414e-b491-17a5c2f8fe84 admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:40.556302 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a20a2f-76a7-414e-b491-17a5c2f8fe84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:40.557904 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a20a2f-76a7-414e-b491-17a5c2f8fe84 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:40.568676 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1036/2091] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:20:40 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:20:40.572634 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a20a2f-76a7-414e-b491-17a5c2f8fe84 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:40.583428 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a20a2f-76a7-414e-b491-17a5c2f8fe84 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:40.588197 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d00e06-07ba-4a68-88a8-16f80c3b422a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:40.588814 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d00e06-07ba-4a68-88a8-16f80c3b422a admin admin] REQUEST_METHOD: `PUT` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:40.589098 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d00e06-07ba-4a68-88a8-16f80c3b422a admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:40.595900 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-51beca4d-c152-48e2-aa55-35725fb9e9e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7cbef142fa684e63bc8e786b138dd953', 'request_id': 'req-51beca4d-c152-48e2-aa55-35725fb9e9e0'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:40.596029 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d00e06-07ba-4a68-88a8-16f80c3b422a admin admin] PATH_INFO: `/v3/projects/91a9a2ba30f6433d976f82cba18a5373/users/dfbf85b9e5b140ceba1fe0510ee1109f/roles/3255f77192524a16bc405173106871fc` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:40.596227 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-51beca4d-c152-48e2-aa55-35725fb9e9e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7cbef142fa684e63bc8e786b138dd953', 'request_id': 'req-51beca4d-c152-48e2-aa55-35725fb9e9e0'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:40.605781 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d00e06-07ba-4a68-88a8-16f80c3b422a admin admin] RBAC: Authorizing `identity:create_grant(project_id=91a9a2ba30f6433d976f82cba18a5373, user_id=dfbf85b9e5b140ceba1fe0510ee1109f, role_id=3255f77192524a16bc405173106871fc)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:40.608207 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-51beca4d-c152-48e2-aa55-35725fb9e9e0 admin admin] The token cache is being invalidate because project 7cbef142fa684e63bc8e786b138dd953 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:20:40.608434 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d00e06-07ba-4a68-88a8-16f80c3b422a admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:40.608711 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-51beca4d-c152-48e2-aa55-35725fb9e9e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:40.609383 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-51beca4d-c152-48e2-aa55-35725fb9e9e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:40.614577 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1056/2092] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:20:40 2022] DELETE /identity/v3/projects/7cbef142fa684e63bc8e786b138dd953 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:20:40.619560 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51beca4d-c152-48e2-aa55-35725fb9e9e0 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:40.624489 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-51beca4d-c152-48e2-aa55-35725fb9e9e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:40.636223 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51beca4d-c152-48e2-aa55-35725fb9e9e0 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:40.641063 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19475a5-15f8-4597-a121-2cbc1a80e89e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:40.641712 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19475a5-15f8-4597-a121-2cbc1a80e89e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:40.641990 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19475a5-15f8-4597-a121-2cbc1a80e89e admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:40.642259 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19475a5-15f8-4597-a121-2cbc1a80e89e admin admin] PATH_INFO: `/v3/users/f36bf528908a453991bfd0c17a107393` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:40.643626 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19475a5-15f8-4597-a121-2cbc1a80e89e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f36bf528908a453991bfd0c17a107393)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:40.646229 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19475a5-15f8-4597-a121-2cbc1a80e89e admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:40.703985 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1037/2093] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:20:40 2022] PUT /identity/v3/projects/91a9a2ba30f6433d976f82cba18a5373/users/dfbf85b9e5b140ceba1fe0510ee1109f/roles/3255f77192524a16bc405173106871fc => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:20:40.714544 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d32a934-a3f3-4d3a-bfc2-e0eba2de6232 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:40.714830 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d32a934-a3f3-4d3a-bfc2-e0eba2de6232 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:40.715130 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d32a934-a3f3-4d3a-bfc2-e0eba2de6232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:40.741933 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-4d32a934-a3f3-4d3a-bfc2-e0eba2de6232 None None] MFA Rules not processed for user `dfbf85b9e5b140ceba1fe0510ee1109f`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:20:40.743200 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-4d32a934-a3f3-4d3a-bfc2-e0eba2de6232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:40.809624 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:20:40.828836 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-a19475a5-15f8-4597-a121-2cbc1a80e89e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f36bf528908a453991bfd0c17a107393'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:40.837983 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1038/2094] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:20:40 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:20:40.849495 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d32a934-a3f3-4d3a-bfc2-e0eba2de6232 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:40.852092 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-4d32a934-a3f3-4d3a-bfc2-e0eba2de6232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:40.932189 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-a19475a5-15f8-4597-a121-2cbc1a80e89e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f36bf528908a453991bfd0c17a107393'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:40.932864 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-a19475a5-15f8-4597-a121-2cbc1a80e89e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f36bf528908a453991bfd0c17a107393'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:40.956063 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d32a934-a3f3-4d3a-bfc2-e0eba2de6232 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:40.956279 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-a19475a5-15f8-4597-a121-2cbc1a80e89e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f36bf528908a453991bfd0c17a107393'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:40.960976 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30f78bc-c3a9-43e6-b676-6c4ace5cf2ef service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f89089b390c4b6599e54000c0fa082a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:40.961816 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30f78bc-c3a9-43e6-b676-6c4ace5cf2ef service glance] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:40.962553 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30f78bc-c3a9-43e6-b676-6c4ace5cf2ef service glance] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:40.962981 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30f78bc-c3a9-43e6-b676-6c4ace5cf2ef service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:40.973841 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30f78bc-c3a9-43e6-b676-6c4ace5cf2ef service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:40.975308 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-a19475a5-15f8-4597-a121-2cbc1a80e89e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f36bf528908a453991bfd0c17a107393'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:40.976166 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30f78bc-c3a9-43e6-b676-6c4ace5cf2ef service glance] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:40.987429 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1057/2095] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:20:40 2022] DELETE /identity/v3/users/f36bf528908a453991bfd0c17a107393 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:20:40.994879 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1039/2096] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:20:40 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:20:41.459306 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19475a5-15f8-4597-a121-2cbc1a80e89e admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:41.460763 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-a19475a5-15f8-4597-a121-2cbc1a80e89e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:41.499441 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:20:41.528958 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19475a5-15f8-4597-a121-2cbc1a80e89e admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:41.533647 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8c2068-f4d9-4400-ac70-739df259b374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:41.534910 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8c2068-f4d9-4400-ac70-739df259b374 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:41.535709 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8c2068-f4d9-4400-ac70-739df259b374 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:41.535991 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8c2068-f4d9-4400-ac70-739df259b374 admin admin] PATH_INFO: `/v3/projects/66cb8c8f830b40209e29cf866b250b63` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:41.537164 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8c2068-f4d9-4400-ac70-739df259b374 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66cb8c8f830b40209e29cf866b250b63)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:41.539027 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8c2068-f4d9-4400-ac70-739df259b374 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:41.682031 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-7e8c2068-f4d9-4400-ac70-739df259b374 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66cb8c8f830b40209e29cf866b250b63', 'request_id': 'req-7e8c2068-f4d9-4400-ac70-739df259b374'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:41.743180 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-7e8c2068-f4d9-4400-ac70-739df259b374 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66cb8c8f830b40209e29cf866b250b63', 'request_id': 'req-7e8c2068-f4d9-4400-ac70-739df259b374'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:41.749466 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-7e8c2068-f4d9-4400-ac70-739df259b374 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66cb8c8f830b40209e29cf866b250b63', 'request_id': 'req-7e8c2068-f4d9-4400-ac70-739df259b374'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:41.760998 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-7e8c2068-f4d9-4400-ac70-739df259b374 admin admin] The token cache is being invalidate because project 66cb8c8f830b40209e29cf866b250b63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:20:41.761290 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-7e8c2068-f4d9-4400-ac70-739df259b374 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:41.761934 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-7e8c2068-f4d9-4400-ac70-739df259b374 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:41.763465 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1058/2097] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:20:41 2022] DELETE /identity/v3/projects/66cb8c8f830b40209e29cf866b250b63 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:20:41.768496 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1be167-c4cc-4cb3-877d-be4ea64a2d19 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:41.768963 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1be167-c4cc-4cb3-877d-be4ea64a2d19 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:41.769212 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1be167-c4cc-4cb3-877d-be4ea64a2d19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:41.790089 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-7e1be167-c4cc-4cb3-877d-be4ea64a2d19 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:20:41.791654 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-7e1be167-c4cc-4cb3-877d-be4ea64a2d19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:41.867504 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1040/2098] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:20:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:20:41.875039 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681b2f91-4154-416d-b03b-efc143f24dec None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:41.875328 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681b2f91-4154-416d-b03b-efc143f24dec None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:41.875634 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681b2f91-4154-416d-b03b-efc143f24dec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:41.896288 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-681b2f91-4154-416d-b03b-efc143f24dec None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:20:41.922342 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-681b2f91-4154-416d-b03b-efc143f24dec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:41.922342 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1059/2099] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:20:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:20:41.922776 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1be167-c4cc-4cb3-877d-be4ea64a2d19 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:41.922776 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1be167-c4cc-4cb3-877d-be4ea64a2d19 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:41.928019 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b06f832-abf3-47a9-baa6-54a4dead1bf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:41.929090 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b06f832-abf3-47a9-baa6-54a4dead1bf2 admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:41.929383 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b06f832-abf3-47a9-baa6-54a4dead1bf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:41.929658 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b06f832-abf3-47a9-baa6-54a4dead1bf2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:41.930890 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b06f832-abf3-47a9-baa6-54a4dead1bf2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:20:41.931054 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b06f832-abf3-47a9-baa6-54a4dead1bf2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:41.932794 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b06f832-abf3-47a9-baa6-54a4dead1bf2 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:41.950987 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1041/2100] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Aug 25 03:20:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:20:41.951818 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681b2f91-4154-416d-b03b-efc143f24dec None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:41.963446 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681b2f91-4154-416d-b03b-efc143f24dec None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:41.968722 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4106f3-c7ed-4193-b4fa-d7d47bc5f01a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:41.969417 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4106f3-c7ed-4193-b4fa-d7d47bc5f01a admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:41.969754 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4106f3-c7ed-4193-b4fa-d7d47bc5f01a admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:41.970031 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4106f3-c7ed-4193-b4fa-d7d47bc5f01a admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:41.970855 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4106f3-c7ed-4193-b4fa-d7d47bc5f01a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:41.973438 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4106f3-c7ed-4193-b4fa-d7d47bc5f01a admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:42.059760 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1060/2101] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:20:41 2022] POST /identity/v3/projects => generated 375 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:20:42.063973 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b06f832-abf3-47a9-baa6-54a4dead1bf2 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:42.081011 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b06f832-abf3-47a9-baa6-54a4dead1bf2 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:42.135173 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4106f3-c7ed-4193-b4fa-d7d47bc5f01a admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:42.137997 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-ee4106f3-c7ed-4193-b4fa-d7d47bc5f01a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:42.138618 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:20:42.145159 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fec4956-601a-4267-9808-f132719b14e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:42.145782 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fec4956-601a-4267-9808-f132719b14e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:42.146067 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fec4956-601a-4267-9808-f132719b14e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:42.146371 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fec4956-601a-4267-9808-f132719b14e3 admin admin] PATH_INFO: `/v3/users` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:42.147257 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fec4956-601a-4267-9808-f132719b14e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:42.149926 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fec4956-601a-4267-9808-f132719b14e3 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:42.209586 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4106f3-c7ed-4193-b4fa-d7d47bc5f01a admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:42.215079 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a112e41-c9d3-413f-a49d-444c7536e798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:42.215709 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a112e41-c9d3-413f-a49d-444c7536e798 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:42.215986 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a112e41-c9d3-413f-a49d-444c7536e798 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:42.216251 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a112e41-c9d3-413f-a49d-444c7536e798 admin admin] PATH_INFO: `/v3/users/5adc372e2fdc45ec97c02ca7f90f1ffc` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:42.217532 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a112e41-c9d3-413f-a49d-444c7536e798 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5adc372e2fdc45ec97c02ca7f90f1ffc)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:42.219551 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a112e41-c9d3-413f-a49d-444c7536e798 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:42.245639 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1042/2102] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:20:42 2022] POST /identity/v3/users => generated 288 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:20:42.246957 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fec4956-601a-4267-9808-f132719b14e3 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:42.263449 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fec4956-601a-4267-9808-f132719b14e3 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:42.271032 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377d058e-ff3a-4429-a164-ea32c3436a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:42.271832 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d058e-ff3a-4429-a164-ea32c3436a73 admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:42.272304 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d058e-ff3a-4429-a164-ea32c3436a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:42.272887 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d058e-ff3a-4429-a164-ea32c3436a73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:42.273701 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377d058e-ff3a-4429-a164-ea32c3436a73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:42.276448 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377d058e-ff3a-4429-a164-ea32c3436a73 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:42.291894 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1043/2103] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:20:42 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:20:42.295074 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377d058e-ff3a-4429-a164-ea32c3436a73 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:42.309685 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377d058e-ff3a-4429-a164-ea32c3436a73 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:42.316643 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43fc5ff-a788-45bc-aa8d-aa38e2333e08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:42.317829 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43fc5ff-a788-45bc-aa8d-aa38e2333e08 admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:42.318349 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43fc5ff-a788-45bc-aa8d-aa38e2333e08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:42.318922 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43fc5ff-a788-45bc-aa8d-aa38e2333e08 admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:42.320055 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43fc5ff-a788-45bc-aa8d-aa38e2333e08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:42.322895 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43fc5ff-a788-45bc-aa8d-aa38e2333e08 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:42.342298 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1044/2104] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:20:42 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:20:42.347086 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43fc5ff-a788-45bc-aa8d-aa38e2333e08 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:42.367139 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43fc5ff-a788-45bc-aa8d-aa38e2333e08 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:42.367898 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c260143-ff70-4d0a-8df1-df421a4ccdc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:42.368781 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c260143-ff70-4d0a-8df1-df421a4ccdc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:42.369193 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c260143-ff70-4d0a-8df1-df421a4ccdc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:42.369591 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c260143-ff70-4d0a-8df1-df421a4ccdc0 admin admin] PATH_INFO: `/v3/projects/cbdf6fc20d7d4fcc84dc6155eeb8ac12/users/5011c09a774a4adca67051ef5e0c34f5/roles/3255f77192524a16bc405173106871fc` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:42.399458 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c260143-ff70-4d0a-8df1-df421a4ccdc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbdf6fc20d7d4fcc84dc6155eeb8ac12, user_id=5011c09a774a4adca67051ef5e0c34f5, role_id=3255f77192524a16bc405173106871fc)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:42.402930 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c260143-ff70-4d0a-8df1-df421a4ccdc0 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:42.424904 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:20:42.434152 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-3a112e41-c9d3-413f-a49d-444c7536e798 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5adc372e2fdc45ec97c02ca7f90f1ffc'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:42.505186 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1045/2105] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:20:42 2022] PUT /identity/v3/projects/cbdf6fc20d7d4fcc84dc6155eeb8ac12/users/5011c09a774a4adca67051ef5e0c34f5/roles/3255f77192524a16bc405173106871fc => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:20:42.515454 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fcfdf7-58df-4e4c-9b7b-655cfd6668bc None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:42.515748 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fcfdf7-58df-4e4c-9b7b-655cfd6668bc None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:42.516030 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fcfdf7-58df-4e4c-9b7b-655cfd6668bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:42.527589 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-3a112e41-c9d3-413f-a49d-444c7536e798 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5adc372e2fdc45ec97c02ca7f90f1ffc'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:42.528556 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-3a112e41-c9d3-413f-a49d-444c7536e798 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5adc372e2fdc45ec97c02ca7f90f1ffc'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:42.537939 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-3a112e41-c9d3-413f-a49d-444c7536e798 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5adc372e2fdc45ec97c02ca7f90f1ffc'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:42.547163 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-89fcfdf7-58df-4e4c-9b7b-655cfd6668bc None None] MFA Rules not processed for user `5011c09a774a4adca67051ef5e0c34f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:20:42.548805 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-89fcfdf7-58df-4e4c-9b7b-655cfd6668bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:42.559542 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-3a112e41-c9d3-413f-a49d-444c7536e798 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5adc372e2fdc45ec97c02ca7f90f1ffc'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:42.567994 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1061/2106] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:20:42 2022] DELETE /identity/v3/users/5adc372e2fdc45ec97c02ca7f90f1ffc => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:20:42.573778 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baec5c7d-50c5-4e2c-87e7-8602683965ff None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:42.574269 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baec5c7d-50c5-4e2c-87e7-8602683965ff None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:42.574967 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baec5c7d-50c5-4e2c-87e7-8602683965ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:42.616521 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-baec5c7d-50c5-4e2c-87e7-8602683965ff None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:20:42.617791 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-baec5c7d-50c5-4e2c-87e7-8602683965ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:42.647058 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1046/2107] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:20:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:20:42.659339 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fcfdf7-58df-4e4c-9b7b-655cfd6668bc None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:42.661576 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-89fcfdf7-58df-4e4c-9b7b-655cfd6668bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:42.696534 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1062/2108] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:20:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:20:42.703712 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baec5c7d-50c5-4e2c-87e7-8602683965ff None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:42.705649 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-baec5c7d-50c5-4e2c-87e7-8602683965ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:42.726988 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fcfdf7-58df-4e4c-9b7b-655cfd6668bc None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:42.775024 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:20:42.777621 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baec5c7d-50c5-4e2c-87e7-8602683965ff None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:42.784826 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5910f1f0-614e-4146-8628-af3d17a151e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60193ba9e1c14df3bc9eebcc6f28f2d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:42.785597 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5910f1f0-614e-4146-8628-af3d17a151e8 service cinder] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:42.788042 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5910f1f0-614e-4146-8628-af3d17a151e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:42.788042 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5910f1f0-614e-4146-8628-af3d17a151e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:42.788327 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58698a08-03f1-4423-ace2-47a99d40f8f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:42.788327 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58698a08-03f1-4423-ace2-47a99d40f8f4 service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:42.788327 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58698a08-03f1-4423-ace2-47a99d40f8f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:42.788327 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58698a08-03f1-4423-ace2-47a99d40f8f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:42.797963 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58698a08-03f1-4423-ace2-47a99d40f8f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:42.799576 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58698a08-03f1-4423-ace2-47a99d40f8f4 service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:42.809524 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1063/2109] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:20:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 25 03:20:42.849303 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5910f1f0-614e-4146-8628-af3d17a151e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:42.850928 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5910f1f0-614e-4146-8628-af3d17a151e8 service cinder] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:42.860124 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1047/2110] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:20:42 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:20:43.277035 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58698a08-03f1-4423-ace2-47a99d40f8f4 service neutron] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:43.282908 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-58698a08-03f1-4423-ace2-47a99d40f8f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:43.305685 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58698a08-03f1-4423-ace2-47a99d40f8f4 service neutron] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:43.321156 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054e444b-d530-4015-980d-ecbc26a0fe18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:43.321156 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054e444b-d530-4015-980d-ecbc26a0fe18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:43.321156 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054e444b-d530-4015-980d-ecbc26a0fe18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:43.321156 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054e444b-d530-4015-980d-ecbc26a0fe18 admin admin] PATH_INFO: `/v3/projects/1cf782e16e5d4d6bbcf56bec07a1c506` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:43.321992 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054e444b-d530-4015-980d-ecbc26a0fe18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cf782e16e5d4d6bbcf56bec07a1c506)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:43.324849 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054e444b-d530-4015-980d-ecbc26a0fe18 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:43.493609 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-054e444b-d530-4015-980d-ecbc26a0fe18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cf782e16e5d4d6bbcf56bec07a1c506', 'request_id': 'req-054e444b-d530-4015-980d-ecbc26a0fe18'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:43.495739 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:20:43.589157 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-054e444b-d530-4015-980d-ecbc26a0fe18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cf782e16e5d4d6bbcf56bec07a1c506', 'request_id': 'req-054e444b-d530-4015-980d-ecbc26a0fe18'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:43.596088 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-054e444b-d530-4015-980d-ecbc26a0fe18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cf782e16e5d4d6bbcf56bec07a1c506', 'request_id': 'req-054e444b-d530-4015-980d-ecbc26a0fe18'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:43.607912 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-054e444b-d530-4015-980d-ecbc26a0fe18 admin admin] The token cache is being invalidate because project 1cf782e16e5d4d6bbcf56bec07a1c506 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:20:43.608196 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-054e444b-d530-4015-980d-ecbc26a0fe18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:43.608850 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-054e444b-d530-4015-980d-ecbc26a0fe18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:43.610416 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1064/2111] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:20:43 2022] DELETE /identity/v3/projects/1cf782e16e5d4d6bbcf56bec07a1c506 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:20:43.618158 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5910f1f0-614e-4146-8628-af3d17a151e8 service cinder] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:43.632814 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5910f1f0-614e-4146-8628-af3d17a151e8 service cinder] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:43.713044 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf8d9bd-efd9-4e54-bdc5-cc522a5b36b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:43.713459 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf8d9bd-efd9-4e54-bdc5-cc522a5b36b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:43.713753 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf8d9bd-efd9-4e54-bdc5-cc522a5b36b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:43.714055 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf8d9bd-efd9-4e54-bdc5-cc522a5b36b0 admin admin] PATH_INFO: `/v3/users/1196f51d38e043219bfad6215fe71004` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:43.715248 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf8d9bd-efd9-4e54-bdc5-cc522a5b36b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1196f51d38e043219bfad6215fe71004)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:43.717174 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf8d9bd-efd9-4e54-bdc5-cc522a5b36b0 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:43.947016 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-bdf8d9bd-efd9-4e54-bdc5-cc522a5b36b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1196f51d38e043219bfad6215fe71004'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:44.017390 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-bdf8d9bd-efd9-4e54-bdc5-cc522a5b36b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1196f51d38e043219bfad6215fe71004'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:44.018015 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-bdf8d9bd-efd9-4e54-bdc5-cc522a5b36b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1196f51d38e043219bfad6215fe71004'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:44.028022 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-bdf8d9bd-efd9-4e54-bdc5-cc522a5b36b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1196f51d38e043219bfad6215fe71004'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:44.031846 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-bdf8d9bd-efd9-4e54-bdc5-cc522a5b36b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1196f51d38e043219bfad6215fe71004'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:44.051202 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1048/2112] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:20:43 2022] DELETE /identity/v3/users/1196f51d38e043219bfad6215fe71004 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:20:44.436561 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054e444b-d530-4015-980d-ecbc26a0fe18 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:44.438108 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-054e444b-d530-4015-980d-ecbc26a0fe18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:44.452330 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054e444b-d530-4015-980d-ecbc26a0fe18 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:44.513816 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3222afc3-ac0d-4d9e-88ba-a4125e38b49a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:44.514510 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3222afc3-ac0d-4d9e-88ba-a4125e38b49a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:44.514793 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3222afc3-ac0d-4d9e-88ba-a4125e38b49a admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:44.515068 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3222afc3-ac0d-4d9e-88ba-a4125e38b49a admin admin] PATH_INFO: `/v3/projects/5c3999fd375745c29e8dbd72a22fa5dc` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:44.516217 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3222afc3-ac0d-4d9e-88ba-a4125e38b49a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c3999fd375745c29e8dbd72a22fa5dc)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:44.518134 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3222afc3-ac0d-4d9e-88ba-a4125e38b49a admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:44.633011 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-3222afc3-ac0d-4d9e-88ba-a4125e38b49a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c3999fd375745c29e8dbd72a22fa5dc', 'request_id': 'req-3222afc3-ac0d-4d9e-88ba-a4125e38b49a'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:44.720753 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-3222afc3-ac0d-4d9e-88ba-a4125e38b49a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c3999fd375745c29e8dbd72a22fa5dc', 'request_id': 'req-3222afc3-ac0d-4d9e-88ba-a4125e38b49a'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:44.728155 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-3222afc3-ac0d-4d9e-88ba-a4125e38b49a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c3999fd375745c29e8dbd72a22fa5dc', 'request_id': 'req-3222afc3-ac0d-4d9e-88ba-a4125e38b49a'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:44.741460 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-3222afc3-ac0d-4d9e-88ba-a4125e38b49a admin admin] The token cache is being invalidate because project 5c3999fd375745c29e8dbd72a22fa5dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:20:44.741779 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-3222afc3-ac0d-4d9e-88ba-a4125e38b49a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:44.742767 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-3222afc3-ac0d-4d9e-88ba-a4125e38b49a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:44.744250 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1065/2113] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:20:44 2022] DELETE /identity/v3/projects/5c3999fd375745c29e8dbd72a22fa5dc => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:20:44.751293 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c68606-6d29-49d3-9715-7893429d2b66 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:44.751521 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c68606-6d29-49d3-9715-7893429d2b66 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:44.751795 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c68606-6d29-49d3-9715-7893429d2b66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:44.771966 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-76c68606-6d29-49d3-9715-7893429d2b66 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:20:44.773162 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-76c68606-6d29-49d3-9715-7893429d2b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:44.786319 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:20:44.833588 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1049/2114] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:20:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:20:44.842195 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7429fdc5-f204-4005-81c9-23a4c2116837 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:44.842509 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7429fdc5-f204-4005-81c9-23a4c2116837 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:44.842789 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7429fdc5-f204-4005-81c9-23a4c2116837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:44.862497 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-7429fdc5-f204-4005-81c9-23a4c2116837 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:20:44.864296 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-7429fdc5-f204-4005-81c9-23a4c2116837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:44.872256 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1066/2115] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:20:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:20:44.876574 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c68606-6d29-49d3-9715-7893429d2b66 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:44.887101 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c68606-6d29-49d3-9715-7893429d2b66 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:44.891768 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acee90d3-799b-48a2-99c1-353e2151c2ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:44.892394 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acee90d3-799b-48a2-99c1-353e2151c2ad admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:44.892656 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acee90d3-799b-48a2-99c1-353e2151c2ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:44.892928 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acee90d3-799b-48a2-99c1-353e2151c2ad admin admin] PATH_INFO: `/v3/domains` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:44.893576 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acee90d3-799b-48a2-99c1-353e2151c2ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:20:44.893873 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acee90d3-799b-48a2-99c1-353e2151c2ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:44.895548 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acee90d3-799b-48a2-99c1-353e2151c2ad admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:44.907415 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1050/2116] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Aug 25 03:20:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:20:44.912206 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7429fdc5-f204-4005-81c9-23a4c2116837 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:44.922458 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7429fdc5-f204-4005-81c9-23a4c2116837 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:44.927722 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fac889a-cfbf-4707-9de0-cad2c063466b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:44.928328 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fac889a-cfbf-4707-9de0-cad2c063466b admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:44.928612 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fac889a-cfbf-4707-9de0-cad2c063466b admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:44.930477 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fac889a-cfbf-4707-9de0-cad2c063466b admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:44.930477 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fac889a-cfbf-4707-9de0-cad2c063466b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:44.931432 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fac889a-cfbf-4707-9de0-cad2c063466b admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:45.019276 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1067/2117] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:20:44 2022] POST /identity/v3/projects => generated 373 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:20:45.023494 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acee90d3-799b-48a2-99c1-353e2151c2ad admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:45.033759 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acee90d3-799b-48a2-99c1-353e2151c2ad admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:45.092859 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d079cad9-9098-4cb4-abcd-8fed1909cc61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:45.093477 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d079cad9-9098-4cb4-abcd-8fed1909cc61 admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:45.093791 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d079cad9-9098-4cb4-abcd-8fed1909cc61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:45.094067 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d079cad9-9098-4cb4-abcd-8fed1909cc61 admin admin] PATH_INFO: `/v3/users` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:45.094779 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d079cad9-9098-4cb4-abcd-8fed1909cc61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:45.096749 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d079cad9-9098-4cb4-abcd-8fed1909cc61 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:45.227585 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1051/2118] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:20:45 2022] POST /identity/v3/users => generated 287 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:20:45.229145 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fac889a-cfbf-4707-9de0-cad2c063466b admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:45.240524 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fac889a-cfbf-4707-9de0-cad2c063466b admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:45.245190 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f79a8a3-402f-4935-8b90-2a8c06896c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:45.245899 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f79a8a3-402f-4935-8b90-2a8c06896c4a admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:45.246244 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f79a8a3-402f-4935-8b90-2a8c06896c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:45.246610 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f79a8a3-402f-4935-8b90-2a8c06896c4a admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:45.247379 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f79a8a3-402f-4935-8b90-2a8c06896c4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:45.249077 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f79a8a3-402f-4935-8b90-2a8c06896c4a admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:45.260446 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1068/2119] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:20:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:20:45.264541 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d079cad9-9098-4cb4-abcd-8fed1909cc61 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:45.275142 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d079cad9-9098-4cb4-abcd-8fed1909cc61 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:45.280151 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262b1121-1194-4185-a696-5a3e0104f80b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:45.280863 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262b1121-1194-4185-a696-5a3e0104f80b admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:45.281184 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262b1121-1194-4185-a696-5a3e0104f80b admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:45.281523 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262b1121-1194-4185-a696-5a3e0104f80b admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:45.282246 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262b1121-1194-4185-a696-5a3e0104f80b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:45.284047 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262b1121-1194-4185-a696-5a3e0104f80b admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:45.296331 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1052/2120] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:20:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 25 03:20:45.301976 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f79a8a3-402f-4935-8b90-2a8c06896c4a admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:45.336959 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f79a8a3-402f-4935-8b90-2a8c06896c4a admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:45.336959 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aae9765-b28d-4b8d-8f4a-0255ca6aafc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:45.336959 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aae9765-b28d-4b8d-8f4a-0255ca6aafc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:45.336959 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aae9765-b28d-4b8d-8f4a-0255ca6aafc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:45.337302 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aae9765-b28d-4b8d-8f4a-0255ca6aafc7 admin admin] PATH_INFO: `/v3/projects/aade369816e0473e856734c4233a69d6/users/cf09092a1fb24f44a65d702a1db73f59/roles/3255f77192524a16bc405173106871fc` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:45.353263 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aae9765-b28d-4b8d-8f4a-0255ca6aafc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aade369816e0473e856734c4233a69d6, user_id=cf09092a1fb24f44a65d702a1db73f59, role_id=3255f77192524a16bc405173106871fc)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:45.356022 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aae9765-b28d-4b8d-8f4a-0255ca6aafc7 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:45.418805 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1069/2121] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:20:45 2022] PUT /identity/v3/projects/aade369816e0473e856734c4233a69d6/users/cf09092a1fb24f44a65d702a1db73f59/roles/3255f77192524a16bc405173106871fc => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:20:45.431167 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495f8e8b-ff8a-4feb-9f62-f00e0036665f None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:45.431455 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495f8e8b-ff8a-4feb-9f62-f00e0036665f None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:45.431731 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495f8e8b-ff8a-4feb-9f62-f00e0036665f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:45.460820 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-495f8e8b-ff8a-4feb-9f62-f00e0036665f None None] MFA Rules not processed for user `cf09092a1fb24f44a65d702a1db73f59`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:20:45.461791 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-495f8e8b-ff8a-4feb-9f62-f00e0036665f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:45.530703 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1053/2122] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:20:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:20:45.535801 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aae9765-b28d-4b8d-8f4a-0255ca6aafc7 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:45.541538 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:INSERT=4,keystone:DELETE=10,keystone:SELECT=86 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:20:45.601518 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aae9765-b28d-4b8d-8f4a-0255ca6aafc7 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:45.606505 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56e6d34-678e-4af5-9a61-40d88f17eb87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:45.607471 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56e6d34-678e-4af5-9a61-40d88f17eb87 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:45.607809 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56e6d34-678e-4af5-9a61-40d88f17eb87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:45.608084 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56e6d34-678e-4af5-9a61-40d88f17eb87 admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:45.608706 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56e6d34-678e-4af5-9a61-40d88f17eb87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:45.610643 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56e6d34-678e-4af5-9a61-40d88f17eb87 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:45.688911 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1070/2123] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:20:45 2022] POST /identity/v3/projects => generated 377 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:20:45.693210 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495f8e8b-ff8a-4feb-9f62-f00e0036665f None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:45.704721 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495f8e8b-ff8a-4feb-9f62-f00e0036665f None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:45.716844 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:20:45.767921 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3786bf71-667d-4561-8797-24fde88247d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:45.768567 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3786bf71-667d-4561-8797-24fde88247d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:45.768847 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3786bf71-667d-4561-8797-24fde88247d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:45.769121 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3786bf71-667d-4561-8797-24fde88247d4 admin admin] PATH_INFO: `/v3/users` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:45.770471 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3786bf71-667d-4561-8797-24fde88247d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:45.772420 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3786bf71-667d-4561-8797-24fde88247d4 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:45.868530 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1054/2124] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:20:45 2022] POST /identity/v3/users => generated 289 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:20:45.870149 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56e6d34-678e-4af5-9a61-40d88f17eb87 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:45.882792 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56e6d34-678e-4af5-9a61-40d88f17eb87 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:45.889625 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46c1cd7-4d41-4b88-b590-a6c36b105b75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:45.890311 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46c1cd7-4d41-4b88-b590-a6c36b105b75 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:45.890588 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46c1cd7-4d41-4b88-b590-a6c36b105b75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:45.890862 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46c1cd7-4d41-4b88-b590-a6c36b105b75 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:45.891516 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46c1cd7-4d41-4b88-b590-a6c36b105b75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:45.893110 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46c1cd7-4d41-4b88-b590-a6c36b105b75 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:45.910860 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1071/2125] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:20:45 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:20:45.917007 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3786bf71-667d-4561-8797-24fde88247d4 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:45.931602 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3786bf71-667d-4561-8797-24fde88247d4 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:45.936271 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1e2ce9-a404-4108-8180-9c7bcf5b162c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:45.936874 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1e2ce9-a404-4108-8180-9c7bcf5b162c admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:45.937158 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1e2ce9-a404-4108-8180-9c7bcf5b162c admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:45.937429 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1e2ce9-a404-4108-8180-9c7bcf5b162c admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:45.938077 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1e2ce9-a404-4108-8180-9c7bcf5b162c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:45.939741 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1e2ce9-a404-4108-8180-9c7bcf5b162c admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:45.950658 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1055/2126] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:20:45 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:20:45.954889 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46c1cd7-4d41-4b88-b590-a6c36b105b75 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:45.965600 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46c1cd7-4d41-4b88-b590-a6c36b105b75 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:45.970292 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c633b0f9-3e56-4960-ac2a-0f996bed8681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:45.970991 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c633b0f9-3e56-4960-ac2a-0f996bed8681 admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:45.971268 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c633b0f9-3e56-4960-ac2a-0f996bed8681 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:45.971542 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c633b0f9-3e56-4960-ac2a-0f996bed8681 admin admin] PATH_INFO: `/v3/projects/720e3ad46a5c44a99b73599e5ea304c6/users/34701ff69166436bbdad110b841b772f/roles/3255f77192524a16bc405173106871fc` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:45.985629 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c633b0f9-3e56-4960-ac2a-0f996bed8681 admin admin] RBAC: Authorizing `identity:create_grant(project_id=720e3ad46a5c44a99b73599e5ea304c6, user_id=34701ff69166436bbdad110b841b772f, role_id=3255f77192524a16bc405173106871fc)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:45.988522 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c633b0f9-3e56-4960-ac2a-0f996bed8681 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:46.109706 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1072/2127] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:20:45 2022] PUT /identity/v3/projects/720e3ad46a5c44a99b73599e5ea304c6/users/34701ff69166436bbdad110b841b772f/roles/3255f77192524a16bc405173106871fc => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:20:46.120600 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1170ccc4-8523-41a2-bc5b-1bf1cedfaa4e None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:46.121088 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1170ccc4-8523-41a2-bc5b-1bf1cedfaa4e None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:46.121198 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1170ccc4-8523-41a2-bc5b-1bf1cedfaa4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:46.150721 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-1170ccc4-8523-41a2-bc5b-1bf1cedfaa4e None None] MFA Rules not processed for user `34701ff69166436bbdad110b841b772f`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:20:46.151970 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-1170ccc4-8523-41a2-bc5b-1bf1cedfaa4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:46.221973 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1056/2128] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:20:46 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:20:46.226491 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c633b0f9-3e56-4960-ac2a-0f996bed8681 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:46.288885 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c633b0f9-3e56-4960-ac2a-0f996bed8681 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:46.293509 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b895829d-e6ce-4e63-9a16-6edcb04d1fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:46.294143 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b895829d-e6ce-4e63-9a16-6edcb04d1fc7 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:46.294475 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b895829d-e6ce-4e63-9a16-6edcb04d1fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:46.294753 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b895829d-e6ce-4e63-9a16-6edcb04d1fc7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:46.295383 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b895829d-e6ce-4e63-9a16-6edcb04d1fc7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:46.297276 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b895829d-e6ce-4e63-9a16-6edcb04d1fc7 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:46.369101 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1073/2129] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:20:46 2022] POST /identity/v3/projects => generated 375 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:20:46.373307 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1170ccc4-8523-41a2-bc5b-1bf1cedfaa4e None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:46.385131 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1170ccc4-8523-41a2-bc5b-1bf1cedfaa4e None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:46.454125 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ed6985-9ba2-44dc-92a3-ef1fb364d9bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:46.454855 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ed6985-9ba2-44dc-92a3-ef1fb364d9bd admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:46.455212 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ed6985-9ba2-44dc-92a3-ef1fb364d9bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:46.455560 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ed6985-9ba2-44dc-92a3-ef1fb364d9bd admin admin] PATH_INFO: `/v3/users` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:46.456262 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ed6985-9ba2-44dc-92a3-ef1fb364d9bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:46.458358 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ed6985-9ba2-44dc-92a3-ef1fb364d9bd admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:46.538626 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1057/2130] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:20:46 2022] POST /identity/v3/users => generated 294 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:20:46.542594 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b895829d-e6ce-4e63-9a16-6edcb04d1fc7 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:46.553439 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b895829d-e6ce-4e63-9a16-6edcb04d1fc7 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:46.558344 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dddecb-a46c-4a9c-a674-58698ac124f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:46.558954 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dddecb-a46c-4a9c-a674-58698ac124f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:46.559245 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dddecb-a46c-4a9c-a674-58698ac124f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:46.559522 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dddecb-a46c-4a9c-a674-58698ac124f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:46.560169 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dddecb-a46c-4a9c-a674-58698ac124f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:46.562102 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dddecb-a46c-4a9c-a674-58698ac124f5 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:46.574076 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1074/2131] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:20:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:20:46.578517 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ed6985-9ba2-44dc-92a3-ef1fb364d9bd admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:46.588802 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ed6985-9ba2-44dc-92a3-ef1fb364d9bd admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:46.593779 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaf4619-0abc-4c83-87ee-e1ab53af1daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:46.594494 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaf4619-0abc-4c83-87ee-e1ab53af1daf admin admin] REQUEST_METHOD: `PUT` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:46.594784 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaf4619-0abc-4c83-87ee-e1ab53af1daf admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:46.598942 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaf4619-0abc-4c83-87ee-e1ab53af1daf admin admin] PATH_INFO: `/v3/projects/b4f24674ab694d7b87213435752487b4/users/9930f948523a4c3fb641da242c014fab/roles/62f19ea272b447ccaab1576cd9bf4fd9` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:46.612093 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:20:46.616754 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaf4619-0abc-4c83-87ee-e1ab53af1daf admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4f24674ab694d7b87213435752487b4, user_id=9930f948523a4c3fb641da242c014fab, role_id=62f19ea272b447ccaab1576cd9bf4fd9)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:46.619639 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaf4619-0abc-4c83-87ee-e1ab53af1daf admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:46.698153 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1058/2132] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:20:46 2022] PUT /identity/v3/projects/b4f24674ab694d7b87213435752487b4/users/9930f948523a4c3fb641da242c014fab/roles/62f19ea272b447ccaab1576cd9bf4fd9 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:20:46.702484 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dddecb-a46c-4a9c-a674-58698ac124f5 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:46.712235 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dddecb-a46c-4a9c-a674-58698ac124f5 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:46.737466 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:20:46.769777 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b224373f-c5a6-4a25-8a71-0275acc51685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:46.770641 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b224373f-c5a6-4a25-8a71-0275acc51685 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:46.770923 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b224373f-c5a6-4a25-8a71-0275acc51685 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:46.771193 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b224373f-c5a6-4a25-8a71-0275acc51685 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:46.771865 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b224373f-c5a6-4a25-8a71-0275acc51685 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:46.773432 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b224373f-c5a6-4a25-8a71-0275acc51685 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:46.792399 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1075/2133] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:20:46 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:20:46.798770 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaf4619-0abc-4c83-87ee-e1ab53af1daf admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:46.816426 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaf4619-0abc-4c83-87ee-e1ab53af1daf admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:46.823945 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee9c069-49dd-4b3c-8518-7ae2b32c09c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:46.825421 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee9c069-49dd-4b3c-8518-7ae2b32c09c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:46.825959 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee9c069-49dd-4b3c-8518-7ae2b32c09c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:46.826551 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee9c069-49dd-4b3c-8518-7ae2b32c09c4 admin admin] PATH_INFO: `/v3/projects/b4f24674ab694d7b87213435752487b4/users/9930f948523a4c3fb641da242c014fab/roles/62f19ea272b447ccaab1576cd9bf4fd9` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:46.829725 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee9c069-49dd-4b3c-8518-7ae2b32c09c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4f24674ab694d7b87213435752487b4, user_id=9930f948523a4c3fb641da242c014fab, role_id=62f19ea272b447ccaab1576cd9bf4fd9)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:46.835217 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee9c069-49dd-4b3c-8518-7ae2b32c09c4 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:46.849536 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1059/2134] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:20:46 2022] PUT /identity/v3/projects/b4f24674ab694d7b87213435752487b4/users/9930f948523a4c3fb641da242c014fab/roles/62f19ea272b447ccaab1576cd9bf4fd9 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:20:46.860992 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a971a1-61bc-438c-98d7-a2d82cea1e4c None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:46.861283 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a971a1-61bc-438c-98d7-a2d82cea1e4c None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:46.861562 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a971a1-61bc-438c-98d7-a2d82cea1e4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:46.890518 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-b5a971a1-61bc-438c-98d7-a2d82cea1e4c None None] MFA Rules not processed for user `9930f948523a4c3fb641da242c014fab`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:20:46.891749 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-b5a971a1-61bc-438c-98d7-a2d82cea1e4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:46.964262 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1076/2135] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:20:46 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:20:47.119584 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee9c069-49dd-4b3c-8518-7ae2b32c09c4 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:47.121915 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-2ee9c069-49dd-4b3c-8518-7ae2b32c09c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:47.189475 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee9c069-49dd-4b3c-8518-7ae2b32c09c4 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:47.193141 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec04b47-4e67-4f73-b544-9b9cd10d57ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60193ba9e1c14df3bc9eebcc6f28f2d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:47.193760 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec04b47-4e67-4f73-b544-9b9cd10d57ba service cinder] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:47.194041 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec04b47-4e67-4f73-b544-9b9cd10d57ba service cinder] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:47.194341 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec04b47-4e67-4f73-b544-9b9cd10d57ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:47.251553 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec04b47-4e67-4f73-b544-9b9cd10d57ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:47.252979 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec04b47-4e67-4f73-b544-9b9cd10d57ba service cinder] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:47.262218 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1060/2136] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:20:47 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:20:51.519349 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a971a1-61bc-438c-98d7-a2d82cea1e4c None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:51.520926 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-b5a971a1-61bc-438c-98d7-a2d82cea1e4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:51.534665 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a971a1-61bc-438c-98d7-a2d82cea1e4c None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:51.538268 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a463d042-653c-49ff-9bca-668f8d8172d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60193ba9e1c14df3bc9eebcc6f28f2d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:51.538909 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a463d042-653c-49ff-9bca-668f8d8172d8 service cinder] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:51.539191 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a463d042-653c-49ff-9bca-668f8d8172d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:51.539466 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a463d042-653c-49ff-9bca-668f8d8172d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:51.603475 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a463d042-653c-49ff-9bca-668f8d8172d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:51.605079 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a463d042-653c-49ff-9bca-668f8d8172d8 service cinder] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:51.614564 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1077/2137] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:20:51 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:20:52.152131 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec04b47-4e67-4f73-b544-9b9cd10d57ba service cinder] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:52.155541 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-aec04b47-4e67-4f73-b544-9b9cd10d57ba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:52.253018 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec04b47-4e67-4f73-b544-9b9cd10d57ba service cinder] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:52.257725 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e517e4-1760-49dd-bc00-b412bc8c8980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:52.258477 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e517e4-1760-49dd-bc00-b412bc8c8980 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:52.258909 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e517e4-1760-49dd-bc00-b412bc8c8980 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:52.259252 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e517e4-1760-49dd-bc00-b412bc8c8980 admin admin] PATH_INFO: `/v3/users/5011c09a774a4adca67051ef5e0c34f5` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:52.260451 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e517e4-1760-49dd-bc00-b412bc8c8980 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5011c09a774a4adca67051ef5e0c34f5)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:52.262496 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e517e4-1760-49dd-bc00-b412bc8c8980 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:52.488731 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:20:52.501222 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-96e517e4-1760-49dd-bc00-b412bc8c8980 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5011c09a774a4adca67051ef5e0c34f5'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:52.571954 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-96e517e4-1760-49dd-bc00-b412bc8c8980 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5011c09a774a4adca67051ef5e0c34f5'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:52.572663 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-96e517e4-1760-49dd-bc00-b412bc8c8980 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5011c09a774a4adca67051ef5e0c34f5'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:52.573301 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-96e517e4-1760-49dd-bc00-b412bc8c8980 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5011c09a774a4adca67051ef5e0c34f5'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:52.586972 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-96e517e4-1760-49dd-bc00-b412bc8c8980 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5011c09a774a4adca67051ef5e0c34f5'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:52.599144 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1061/2138] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:20:52 2022] DELETE /identity/v3/users/5011c09a774a4adca67051ef5e0c34f5 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:20:52.603914 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ca79b0-c4a5-4084-8937-b25d53d1e07f None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:52.605630 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ca79b0-c4a5-4084-8937-b25d53d1e07f None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:52.605630 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ca79b0-c4a5-4084-8937-b25d53d1e07f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:52.623675 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-29ca79b0-c4a5-4084-8937-b25d53d1e07f None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:20:52.624909 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-29ca79b0-c4a5-4084-8937-b25d53d1e07f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:52.669571 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:20:52.684544 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1078/2139] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:20:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:20:52.694995 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e517e4-1760-49dd-bc00-b412bc8c8980 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:52.696515 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-96e517e4-1760-49dd-bc00-b412bc8c8980 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:52.763459 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e517e4-1760-49dd-bc00-b412bc8c8980 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:52.771316 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9582e6-f4db-4fca-98b9-d361929d8f1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:52.771316 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9582e6-f4db-4fca-98b9-d361929d8f1a service neutron] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:52.771316 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9582e6-f4db-4fca-98b9-d361929d8f1a service neutron] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:52.771316 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9582e6-f4db-4fca-98b9-d361929d8f1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:52.782103 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9582e6-f4db-4fca-98b9-d361929d8f1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:52.783786 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9582e6-f4db-4fca-98b9-d361929d8f1a service neutron] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:52.794120 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1062/2140] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:20:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:20:53.185667 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ca79b0-c4a5-4084-8937-b25d53d1e07f None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:53.187462 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-29ca79b0-c4a5-4084-8937-b25d53d1e07f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:53.284072 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ca79b0-c4a5-4084-8937-b25d53d1e07f None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:53.290378 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8dce56-ed53-4cc5-b0f5-be481a28d893 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60193ba9e1c14df3bc9eebcc6f28f2d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:53.291888 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8dce56-ed53-4cc5-b0f5-be481a28d893 service cinder] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:53.291888 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8dce56-ed53-4cc5-b0f5-be481a28d893 service cinder] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:53.291888 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8dce56-ed53-4cc5-b0f5-be481a28d893 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:53.293675 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-aa8dce56-ed53-4cc5-b0f5-be481a28d893 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:53.380554 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8dce56-ed53-4cc5-b0f5-be481a28d893 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:53.384300 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8dce56-ed53-4cc5-b0f5-be481a28d893 service cinder] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:53.398974 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9582e6-f4db-4fca-98b9-d361929d8f1a service neutron] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:53.401548 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-7e9582e6-f4db-4fca-98b9-d361929d8f1a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:53.408746 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1079/2141] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:20:53 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:20:53.424590 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9582e6-f4db-4fca-98b9-d361929d8f1a service neutron] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:53.437863 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889e53e8-8ce6-42a9-a5b0-49e215c5f66b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:53.438830 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889e53e8-8ce6-42a9-a5b0-49e215c5f66b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:53.439117 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889e53e8-8ce6-42a9-a5b0-49e215c5f66b admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:53.439391 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889e53e8-8ce6-42a9-a5b0-49e215c5f66b admin admin] PATH_INFO: `/v3/projects/cbdf6fc20d7d4fcc84dc6155eeb8ac12` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:53.442686 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889e53e8-8ce6-42a9-a5b0-49e215c5f66b admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbdf6fc20d7d4fcc84dc6155eeb8ac12)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:53.444261 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889e53e8-8ce6-42a9-a5b0-49e215c5f66b admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:53.601056 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-889e53e8-8ce6-42a9-a5b0-49e215c5f66b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbdf6fc20d7d4fcc84dc6155eeb8ac12', 'request_id': 'req-889e53e8-8ce6-42a9-a5b0-49e215c5f66b'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:53.692916 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-889e53e8-8ce6-42a9-a5b0-49e215c5f66b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbdf6fc20d7d4fcc84dc6155eeb8ac12', 'request_id': 'req-889e53e8-8ce6-42a9-a5b0-49e215c5f66b'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:53.700053 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-889e53e8-8ce6-42a9-a5b0-49e215c5f66b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbdf6fc20d7d4fcc84dc6155eeb8ac12', 'request_id': 'req-889e53e8-8ce6-42a9-a5b0-49e215c5f66b'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:53.712874 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-889e53e8-8ce6-42a9-a5b0-49e215c5f66b admin admin] The token cache is being invalidate because project cbdf6fc20d7d4fcc84dc6155eeb8ac12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81995) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:20:53.713169 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-889e53e8-8ce6-42a9-a5b0-49e215c5f66b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:53.713835 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-889e53e8-8ce6-42a9-a5b0-49e215c5f66b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:53.715403 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1063/2142] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:20:53 2022] DELETE /identity/v3/projects/cbdf6fc20d7d4fcc84dc6155eeb8ac12 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:20:53.721299 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c8da08-447a-4498-b768-33c5eaea986e None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:53.721824 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c8da08-447a-4498-b768-33c5eaea986e None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:53.721917 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c8da08-447a-4498-b768-33c5eaea986e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:53.742513 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-a4c8da08-447a-4498-b768-33c5eaea986e None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:20:53.743808 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-a4c8da08-447a-4498-b768-33c5eaea986e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:53.810758 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1080/2143] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:20:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:20:53.819133 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782de474-30d6-46a1-9fb0-bfc3f55b9e9e None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:53.819416 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782de474-30d6-46a1-9fb0-bfc3f55b9e9e None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:53.819687 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782de474-30d6-46a1-9fb0-bfc3f55b9e9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:53.848392 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-782de474-30d6-46a1-9fb0-bfc3f55b9e9e None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:20:53.849765 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-782de474-30d6-46a1-9fb0-bfc3f55b9e9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:53.861755 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1064/2144] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:20:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:20:53.863791 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c8da08-447a-4498-b768-33c5eaea986e None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:53.889322 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c8da08-447a-4498-b768-33c5eaea986e None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:53.911535 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8bf124-50fa-4e3e-be06-cb7f7ea9e680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:53.912179 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8bf124-50fa-4e3e-be06-cb7f7ea9e680 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:53.912465 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8bf124-50fa-4e3e-be06-cb7f7ea9e680 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:53.912743 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8bf124-50fa-4e3e-be06-cb7f7ea9e680 admin admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:53.913411 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8bf124-50fa-4e3e-be06-cb7f7ea9e680 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:20:53.913721 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8bf124-50fa-4e3e-be06-cb7f7ea9e680 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:53.915827 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8bf124-50fa-4e3e-be06-cb7f7ea9e680 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:53.937858 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1081/2145] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Aug 25 03:20:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:20:53.939296 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782de474-30d6-46a1-9fb0-bfc3f55b9e9e None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:53.952862 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782de474-30d6-46a1-9fb0-bfc3f55b9e9e None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:53.961911 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af989ba-0b61-429b-ad63-d92d04a14d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:53.961911 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af989ba-0b61-429b-ad63-d92d04a14d15 admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:53.961911 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af989ba-0b61-429b-ad63-d92d04a14d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:53.961911 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af989ba-0b61-429b-ad63-d92d04a14d15 admin admin] PATH_INFO: `/v3/projects` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:53.962542 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af989ba-0b61-429b-ad63-d92d04a14d15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:53.962633 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af989ba-0b61-429b-ad63-d92d04a14d15 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:54.077236 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1065/2146] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:20:53 2022] POST /identity/v3/projects => generated 375 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:20:54.081613 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8bf124-50fa-4e3e-be06-cb7f7ea9e680 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:54.092056 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8bf124-50fa-4e3e-be06-cb7f7ea9e680 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:54.116737 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:20:54.151622 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7942d867-1e68-466a-ae9c-cec0acde8cf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:54.152248 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7942d867-1e68-466a-ae9c-cec0acde8cf6 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:54.152796 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7942d867-1e68-466a-ae9c-cec0acde8cf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:54.153082 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7942d867-1e68-466a-ae9c-cec0acde8cf6 admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:54.153739 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7942d867-1e68-466a-ae9c-cec0acde8cf6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:54.155696 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7942d867-1e68-466a-ae9c-cec0acde8cf6 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:54.235395 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1082/2147] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:20:54 2022] POST /identity/v3/users => generated 288 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:20:54.240639 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af989ba-0b61-429b-ad63-d92d04a14d15 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:54.251054 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af989ba-0b61-429b-ad63-d92d04a14d15 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:54.255638 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94274a77-feb3-4610-afb0-b7310f1ab508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:54.256290 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94274a77-feb3-4610-afb0-b7310f1ab508 admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:54.256566 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94274a77-feb3-4610-afb0-b7310f1ab508 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:54.256835 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94274a77-feb3-4610-afb0-b7310f1ab508 admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:54.257482 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94274a77-feb3-4610-afb0-b7310f1ab508 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:54.259735 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94274a77-feb3-4610-afb0-b7310f1ab508 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:54.261545 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:20:54.273149 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1066/2148] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:20:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:20:54.277153 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7942d867-1e68-466a-ae9c-cec0acde8cf6 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:54.288091 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7942d867-1e68-466a-ae9c-cec0acde8cf6 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:54.292908 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf607a0-a397-4855-9d94-2daeb9d5ac04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:54.293515 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf607a0-a397-4855-9d94-2daeb9d5ac04 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:54.293830 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf607a0-a397-4855-9d94-2daeb9d5ac04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:54.294105 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf607a0-a397-4855-9d94-2daeb9d5ac04 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:54.294785 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf607a0-a397-4855-9d94-2daeb9d5ac04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:54.296390 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf607a0-a397-4855-9d94-2daeb9d5ac04 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:54.307371 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1083/2149] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:20:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:20:54.315708 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94274a77-feb3-4610-afb0-b7310f1ab508 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:54.325915 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94274a77-feb3-4610-afb0-b7310f1ab508 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:54.330523 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745a1917-c209-4a4c-a418-aa3ef7c913a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:54.331233 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745a1917-c209-4a4c-a418-aa3ef7c913a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:54.331515 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745a1917-c209-4a4c-a418-aa3ef7c913a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:54.331819 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745a1917-c209-4a4c-a418-aa3ef7c913a0 admin admin] PATH_INFO: `/v3/projects/d9bd9bbd655a4a5ba9b4ce00b93ca03b/users/44f3ded2c4a34ecabfbaaac7b71f7aae/roles/3255f77192524a16bc405173106871fc` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:54.347651 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745a1917-c209-4a4c-a418-aa3ef7c913a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9bd9bbd655a4a5ba9b4ce00b93ca03b, user_id=44f3ded2c4a34ecabfbaaac7b71f7aae, role_id=3255f77192524a16bc405173106871fc)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:54.351336 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745a1917-c209-4a4c-a418-aa3ef7c913a0 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:54.413147 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1067/2150] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:20:54 2022] PUT /identity/v3/projects/d9bd9bbd655a4a5ba9b4ce00b93ca03b/users/44f3ded2c4a34ecabfbaaac7b71f7aae/roles/3255f77192524a16bc405173106871fc => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:20:54.423787 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176e11e4-dc53-4081-acc3-5e66dd7b48fd None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:54.424140 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176e11e4-dc53-4081-acc3-5e66dd7b48fd None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:54.424397 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176e11e4-dc53-4081-acc3-5e66dd7b48fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:54.453386 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-176e11e4-dc53-4081-acc3-5e66dd7b48fd None None] MFA Rules not processed for user `44f3ded2c4a34ecabfbaaac7b71f7aae`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:20:54.456262 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-176e11e4-dc53-4081-acc3-5e66dd7b48fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:54.546371 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1084/2151] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:20:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:20:54.565771 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745a1917-c209-4a4c-a418-aa3ef7c913a0 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:54.567132 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-745a1917-c209-4a4c-a418-aa3ef7c913a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:54.631346 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745a1917-c209-4a4c-a418-aa3ef7c913a0 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:54.637407 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccde5e02-7735-4565-bcf3-7df252a80c8e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60193ba9e1c14df3bc9eebcc6f28f2d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:54.637407 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccde5e02-7735-4565-bcf3-7df252a80c8e service cinder] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:54.637407 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccde5e02-7735-4565-bcf3-7df252a80c8e service cinder] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:54.637407 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccde5e02-7735-4565-bcf3-7df252a80c8e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:54.647760 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccde5e02-7735-4565-bcf3-7df252a80c8e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:54.649766 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccde5e02-7735-4565-bcf3-7df252a80c8e service cinder] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:54.672059 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1068/2152] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:20:54 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 25 03:20:55.833748 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c998e0d0-17d5-459d-a9f0-e0c3e5c8f46f None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:55.834304 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c998e0d0-17d5-459d-a9f0-e0c3e5c8f46f None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:55.834304 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c998e0d0-17d5-459d-a9f0-e0c3e5c8f46f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:55.856857 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-c998e0d0-17d5-459d-a9f0-e0c3e5c8f46f None None] MFA Rules not processed for user `2552825edca0418392f877ea9488e36f`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:20:55.858665 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-c998e0d0-17d5-459d-a9f0-e0c3e5c8f46f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:55.912290 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccde5e02-7735-4565-bcf3-7df252a80c8e service cinder] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:55.913789 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-ccde5e02-7735-4565-bcf3-7df252a80c8e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:55.937104 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1085/2153] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:20:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:20:55.980932 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccde5e02-7735-4565-bcf3-7df252a80c8e service cinder] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:55.985561 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddbdcbd-3d99-4b7c-bfb1-1f3b48af64d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:55.986078 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddbdcbd-3d99-4b7c-bfb1-1f3b48af64d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:55.986385 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddbdcbd-3d99-4b7c-bfb1-1f3b48af64d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:55.986660 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddbdcbd-3d99-4b7c-bfb1-1f3b48af64d7 admin admin] PATH_INFO: `/v3/users/b261202b862544599b4dfa8753b98335` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:55.987796 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddbdcbd-3d99-4b7c-bfb1-1f3b48af64d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b261202b862544599b4dfa8753b98335)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:55.990587 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddbdcbd-3d99-4b7c-bfb1-1f3b48af64d7 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:56.174575 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-fddbdcbd-3d99-4b7c-bfb1-1f3b48af64d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b261202b862544599b4dfa8753b98335'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:56.224035 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-fddbdcbd-3d99-4b7c-bfb1-1f3b48af64d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b261202b862544599b4dfa8753b98335'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:56.224873 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-fddbdcbd-3d99-4b7c-bfb1-1f3b48af64d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b261202b862544599b4dfa8753b98335'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:56.225571 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-fddbdcbd-3d99-4b7c-bfb1-1f3b48af64d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b261202b862544599b4dfa8753b98335'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:56.246199 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-fddbdcbd-3d99-4b7c-bfb1-1f3b48af64d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b261202b862544599b4dfa8753b98335'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:56.251057 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:20:56.260095 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1069/2154] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:20:55 2022] DELETE /identity/v3/users/b261202b862544599b4dfa8753b98335 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:20:56.264617 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e275da-73a8-467f-aad6-c6f9adae41dc None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:56.265008 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e275da-73a8-467f-aad6-c6f9adae41dc None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:56.265419 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e275da-73a8-467f-aad6-c6f9adae41dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:56.285076 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-d9e275da-73a8-467f-aad6-c6f9adae41dc None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:20:56.286955 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-d9e275da-73a8-467f-aad6-c6f9adae41dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:56.328341 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:20:56.355898 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1086/2155] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:20:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:20:56.366740 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddbdcbd-3d99-4b7c-bfb1-1f3b48af64d7 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:56.372521 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-fddbdcbd-3d99-4b7c-bfb1-1f3b48af64d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:56.431113 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddbdcbd-3d99-4b7c-bfb1-1f3b48af64d7 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:56.434773 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e85b74f-34a9-46bb-b424-dda5af67ee09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:56.435404 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e85b74f-34a9-46bb-b424-dda5af67ee09 service neutron] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:56.435668 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e85b74f-34a9-46bb-b424-dda5af67ee09 service neutron] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:56.435972 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e85b74f-34a9-46bb-b424-dda5af67ee09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:56.445609 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e85b74f-34a9-46bb-b424-dda5af67ee09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:56.447142 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e85b74f-34a9-46bb-b424-dda5af67ee09 service neutron] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:56.457697 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1070/2156] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:20:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 25 03:20:56.928736 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e275da-73a8-467f-aad6-c6f9adae41dc None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:56.930221 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-d9e275da-73a8-467f-aad6-c6f9adae41dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:56.942844 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e275da-73a8-467f-aad6-c6f9adae41dc None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:56.949357 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be741c8-f74d-4cad-946b-7940398cc2cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:56.950388 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be741c8-f74d-4cad-946b-7940398cc2cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:56.950783 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be741c8-f74d-4cad-946b-7940398cc2cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:56.951133 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be741c8-f74d-4cad-946b-7940398cc2cb admin admin] PATH_INFO: `/v3/projects/2ee064d3e1a24e238761701a200c70e3` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:56.952574 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be741c8-f74d-4cad-946b-7940398cc2cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ee064d3e1a24e238761701a200c70e3)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:56.954973 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be741c8-f74d-4cad-946b-7940398cc2cb admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:57.063723 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-2be741c8-f74d-4cad-946b-7940398cc2cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ee064d3e1a24e238761701a200c70e3', 'request_id': 'req-2be741c8-f74d-4cad-946b-7940398cc2cb'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:57.160971 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-2be741c8-f74d-4cad-946b-7940398cc2cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ee064d3e1a24e238761701a200c70e3', 'request_id': 'req-2be741c8-f74d-4cad-946b-7940398cc2cb'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:57.167398 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-2be741c8-f74d-4cad-946b-7940398cc2cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ee064d3e1a24e238761701a200c70e3', 'request_id': 'req-2be741c8-f74d-4cad-946b-7940398cc2cb'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:57.180630 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-2be741c8-f74d-4cad-946b-7940398cc2cb admin admin] The token cache is being invalidate because project 2ee064d3e1a24e238761701a200c70e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:20:57.180925 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-2be741c8-f74d-4cad-946b-7940398cc2cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:57.181578 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-2be741c8-f74d-4cad-946b-7940398cc2cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:57.183390 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1087/2157] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:20:56 2022] DELETE /identity/v3/projects/2ee064d3e1a24e238761701a200c70e3 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:20:57.187141 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e85b74f-34a9-46bb-b424-dda5af67ee09 service neutron] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:57.197701 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e85b74f-34a9-46bb-b424-dda5af67ee09 service neutron] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:57.256113 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5147db3-0492-4cd0-aaf3-7d6e3b28f3a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:57.256750 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5147db3-0492-4cd0-aaf3-7d6e3b28f3a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:57.257029 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5147db3-0492-4cd0-aaf3-7d6e3b28f3a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:57.257304 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5147db3-0492-4cd0-aaf3-7d6e3b28f3a9 admin admin] PATH_INFO: `/v3/users/84cb260697c14d8798769501a6d7b748` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:57.258459 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5147db3-0492-4cd0-aaf3-7d6e3b28f3a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84cb260697c14d8798769501a6d7b748)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:57.260424 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5147db3-0492-4cd0-aaf3-7d6e3b28f3a9 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:57.471853 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-a5147db3-0492-4cd0-aaf3-7d6e3b28f3a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84cb260697c14d8798769501a6d7b748'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:57.669413 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-a5147db3-0492-4cd0-aaf3-7d6e3b28f3a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84cb260697c14d8798769501a6d7b748'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:57.670391 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-a5147db3-0492-4cd0-aaf3-7d6e3b28f3a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84cb260697c14d8798769501a6d7b748'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:57.671415 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-a5147db3-0492-4cd0-aaf3-7d6e3b28f3a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84cb260697c14d8798769501a6d7b748'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:57.689926 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-a5147db3-0492-4cd0-aaf3-7d6e3b28f3a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84cb260697c14d8798769501a6d7b748'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:57.707552 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1071/2158] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:20:57 2022] DELETE /identity/v3/users/84cb260697c14d8798769501a6d7b748 => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:20:58.330988 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be741c8-f74d-4cad-946b-7940398cc2cb admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:58.333406 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-2be741c8-f74d-4cad-946b-7940398cc2cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:58.386620 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be741c8-f74d-4cad-946b-7940398cc2cb admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:58.463984 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f73480-b895-4fa8-8e81-20651ba919ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:58.464779 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f73480-b895-4fa8-8e81-20651ba919ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:58.465078 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f73480-b895-4fa8-8e81-20651ba919ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:58.465368 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f73480-b895-4fa8-8e81-20651ba919ee admin admin] PATH_INFO: `/v3/projects/63324a77bd4d41409502be3a41fee3c2` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:58.466531 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f73480-b895-4fa8-8e81-20651ba919ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=63324a77bd4d41409502be3a41fee3c2)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:58.468557 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f73480-b895-4fa8-8e81-20651ba919ee admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:58.589497 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-61f73480-b895-4fa8-8e81-20651ba919ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63324a77bd4d41409502be3a41fee3c2', 'request_id': 'req-61f73480-b895-4fa8-8e81-20651ba919ee'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:58.697845 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-61f73480-b895-4fa8-8e81-20651ba919ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63324a77bd4d41409502be3a41fee3c2', 'request_id': 'req-61f73480-b895-4fa8-8e81-20651ba919ee'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:58.704829 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-61f73480-b895-4fa8-8e81-20651ba919ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63324a77bd4d41409502be3a41fee3c2', 'request_id': 'req-61f73480-b895-4fa8-8e81-20651ba919ee'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:58.706450 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:20:58.719098 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-61f73480-b895-4fa8-8e81-20651ba919ee admin admin] The token cache is being invalidate because project 63324a77bd4d41409502be3a41fee3c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:20:58.719839 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-61f73480-b895-4fa8-8e81-20651ba919ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:58.720912 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-61f73480-b895-4fa8-8e81-20651ba919ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:58.722738 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1088/2159] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:20:58 2022] DELETE /identity/v3/projects/63324a77bd4d41409502be3a41fee3c2 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:20:58.727466 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaba9e1-c152-464a-9cca-d8c384f58940 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:58.727932 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaba9e1-c152-464a-9cca-d8c384f58940 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:58.728037 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaba9e1-c152-464a-9cca-d8c384f58940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:58.747687 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-aaaba9e1-c152-464a-9cca-d8c384f58940 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:20:58.748887 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-aaaba9e1-c152-464a-9cca-d8c384f58940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:58.759642 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:20:58.822243 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1072/2160] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:20:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:20:58.825793 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98214eef-0bec-4d47-804d-4e89e711fe6d None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:58.830507 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98214eef-0bec-4d47-804d-4e89e711fe6d None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:58.831141 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98214eef-0bec-4d47-804d-4e89e711fe6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:58.853206 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-98214eef-0bec-4d47-804d-4e89e711fe6d None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:20:58.854723 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-98214eef-0bec-4d47-804d-4e89e711fe6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:58.865322 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1089/2161] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:20:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:20:58.871936 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaba9e1-c152-464a-9cca-d8c384f58940 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:58.880317 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaba9e1-c152-464a-9cca-d8c384f58940 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:58.885187 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee1dc1f-e9b5-47f3-b0c9-d235e5274a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:58.886121 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee1dc1f-e9b5-47f3-b0c9-d235e5274a75 admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:58.886607 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee1dc1f-e9b5-47f3-b0c9-d235e5274a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:58.887073 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee1dc1f-e9b5-47f3-b0c9-d235e5274a75 admin admin] PATH_INFO: `/v3/domains` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:58.887901 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee1dc1f-e9b5-47f3-b0c9-d235e5274a75 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:20:58.888394 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee1dc1f-e9b5-47f3-b0c9-d235e5274a75 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:58.890544 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee1dc1f-e9b5-47f3-b0c9-d235e5274a75 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:58.906142 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1073/2162] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Aug 25 03:20:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:20:58.907655 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98214eef-0bec-4d47-804d-4e89e711fe6d None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:58.917947 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98214eef-0bec-4d47-804d-4e89e711fe6d None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:58.922831 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cc29d2-5dd2-4559-ae2f-19ada5d5365f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:58.923616 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cc29d2-5dd2-4559-ae2f-19ada5d5365f admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:58.923967 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cc29d2-5dd2-4559-ae2f-19ada5d5365f admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:58.925784 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cc29d2-5dd2-4559-ae2f-19ada5d5365f admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:58.925784 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cc29d2-5dd2-4559-ae2f-19ada5d5365f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:58.927606 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cc29d2-5dd2-4559-ae2f-19ada5d5365f admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:59.090406 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1090/2163] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:20:58 2022] POST /identity/v3/projects => generated 355 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:20:59.091970 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee1dc1f-e9b5-47f3-b0c9-d235e5274a75 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:59.105788 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee1dc1f-e9b5-47f3-b0c9-d235e5274a75 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:59.182028 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37f5b6b-12b1-43d8-aced-9e8b583552af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:59.182984 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37f5b6b-12b1-43d8-aced-9e8b583552af admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:59.183292 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37f5b6b-12b1-43d8-aced-9e8b583552af admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:59.183722 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37f5b6b-12b1-43d8-aced-9e8b583552af admin admin] PATH_INFO: `/v3/users` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:59.184520 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37f5b6b-12b1-43d8-aced-9e8b583552af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:59.186869 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37f5b6b-12b1-43d8-aced-9e8b583552af admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:59.289417 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1074/2164] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:20:59 2022] POST /identity/v3/users => generated 278 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:20:59.292021 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cc29d2-5dd2-4559-ae2f-19ada5d5365f admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:59.307763 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cc29d2-5dd2-4559-ae2f-19ada5d5365f admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:59.313416 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba88ba8a-fb05-4246-8d4a-7b2ea03aa649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:59.314180 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba88ba8a-fb05-4246-8d4a-7b2ea03aa649 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:59.314698 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba88ba8a-fb05-4246-8d4a-7b2ea03aa649 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:59.315079 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba88ba8a-fb05-4246-8d4a-7b2ea03aa649 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:59.317600 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba88ba8a-fb05-4246-8d4a-7b2ea03aa649 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:59.317600 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba88ba8a-fb05-4246-8d4a-7b2ea03aa649 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:59.330571 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1091/2165] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:20:59 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:20:59.334271 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37f5b6b-12b1-43d8-aced-9e8b583552af admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:59.345550 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37f5b6b-12b1-43d8-aced-9e8b583552af admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:59.353533 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55308af9-440b-45e9-b1a6-aff8dee0a97d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:59.354309 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55308af9-440b-45e9-b1a6-aff8dee0a97d admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:59.354900 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55308af9-440b-45e9-b1a6-aff8dee0a97d admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:59.355017 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55308af9-440b-45e9-b1a6-aff8dee0a97d admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:59.355855 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55308af9-440b-45e9-b1a6-aff8dee0a97d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:59.357578 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55308af9-440b-45e9-b1a6-aff8dee0a97d admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:59.369827 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1075/2166] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:20:59 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:20:59.374273 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba88ba8a-fb05-4246-8d4a-7b2ea03aa649 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:59.387528 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba88ba8a-fb05-4246-8d4a-7b2ea03aa649 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:59.394156 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7a5dac-5933-4882-a63f-e2cc3a64b42f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:59.395497 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7a5dac-5933-4882-a63f-e2cc3a64b42f admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:59.399205 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7a5dac-5933-4882-a63f-e2cc3a64b42f admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:59.399205 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7a5dac-5933-4882-a63f-e2cc3a64b42f admin admin] PATH_INFO: `/v3/projects/b9c761d2837842d19432cd120d5a0d2c/users/38a68944c88540568508d7b38ecf2200/roles/3255f77192524a16bc405173106871fc` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:59.424722 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7a5dac-5933-4882-a63f-e2cc3a64b42f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9c761d2837842d19432cd120d5a0d2c, user_id=38a68944c88540568508d7b38ecf2200, role_id=3255f77192524a16bc405173106871fc)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:59.428860 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7a5dac-5933-4882-a63f-e2cc3a64b42f admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:59.497786 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55308af9-440b-45e9-b1a6-aff8dee0a97d admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:59.499834 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-55308af9-440b-45e9-b1a6-aff8dee0a97d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:59.513298 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55308af9-440b-45e9-b1a6-aff8dee0a97d admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:59.520268 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1092/2167] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:20:59 2022] PUT /identity/v3/projects/b9c761d2837842d19432cd120d5a0d2c/users/38a68944c88540568508d7b38ecf2200/roles/3255f77192524a16bc405173106871fc => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:20:59.529900 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a728edb-fc58-42c9-8b13-a9ef8de24833 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:59.530644 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a728edb-fc58-42c9-8b13-a9ef8de24833 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:59.531210 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a728edb-fc58-42c9-8b13-a9ef8de24833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:59.666950 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc67149f-163f-4d31-8fd7-068336b693fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:59.668101 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc67149f-163f-4d31-8fd7-068336b693fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:59.668235 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc67149f-163f-4d31-8fd7-068336b693fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:59.668658 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc67149f-163f-4d31-8fd7-068336b693fc admin admin] PATH_INFO: `/v3/users/cf09092a1fb24f44a65d702a1db73f59` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:59.670898 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc67149f-163f-4d31-8fd7-068336b693fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf09092a1fb24f44a65d702a1db73f59)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:59.672996 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc67149f-163f-4d31-8fd7-068336b693fc admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:59.708118 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:20:59.787477 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-0a728edb-fc58-42c9-8b13-a9ef8de24833 None None] MFA Rules not processed for user `38a68944c88540568508d7b38ecf2200`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:20:59.791801 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-0a728edb-fc58-42c9-8b13-a9ef8de24833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:59.883358 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-cc67149f-163f-4d31-8fd7-068336b693fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf09092a1fb24f44a65d702a1db73f59'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:59.911838 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1093/2168] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:20:59 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 383 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:20:59.913049 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a728edb-fc58-42c9-8b13-a9ef8de24833 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:59.929237 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a728edb-fc58-42c9-8b13-a9ef8de24833 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:59.934822 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5cb01a-21fc-4d63-92a8-a6550de9bab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:59.935837 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5cb01a-21fc-4d63-92a8-a6550de9bab9 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:59.936243 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5cb01a-21fc-4d63-92a8-a6550de9bab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:59.936396 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5cb01a-21fc-4d63-92a8-a6550de9bab9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:59.937027 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5cb01a-21fc-4d63-92a8-a6550de9bab9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:59.938968 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5cb01a-21fc-4d63-92a8-a6550de9bab9 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:59.963375 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-cc67149f-163f-4d31-8fd7-068336b693fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf09092a1fb24f44a65d702a1db73f59'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:59.972486 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-cc67149f-163f-4d31-8fd7-068336b693fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf09092a1fb24f44a65d702a1db73f59'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:59.972486 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-cc67149f-163f-4d31-8fd7-068336b693fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf09092a1fb24f44a65d702a1db73f59'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:59.990115 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-cc67149f-163f-4d31-8fd7-068336b693fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf09092a1fb24f44a65d702a1db73f59'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:00.005665 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1076/2169] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:20:59 2022] DELETE /identity/v3/users/cf09092a1fb24f44a65d702a1db73f59 => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:21:00.012373 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f774e6-e76b-4624-b6cb-ea59b44327b7 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:00.012877 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f774e6-e76b-4624-b6cb-ea59b44327b7 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:00.013772 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f774e6-e76b-4624-b6cb-ea59b44327b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:00.034523 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-63f774e6-e76b-4624-b6cb-ea59b44327b7 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:21:00.037150 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-63f774e6-e76b-4624-b6cb-ea59b44327b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:21:00.053496 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1094/2170] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:20:59 2022] POST /identity/v3/projects => generated 355 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:21:00.054577 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5cb01a-21fc-4d63-92a8-a6550de9bab9 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:00.057101 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-6e5cb01a-21fc-4d63-92a8-a6550de9bab9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:21:00.141744 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1077/2171] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:21:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:21:00.155145 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:21:00.159310 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5cb01a-21fc-4d63-92a8-a6550de9bab9 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:00.166113 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f774e6-e76b-4624-b6cb-ea59b44327b7 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:00.166724 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47560323-ec74-43a5-9dc1-7a211cd9da25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:00.167521 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47560323-ec74-43a5-9dc1-7a211cd9da25 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:00.167822 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47560323-ec74-43a5-9dc1-7a211cd9da25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:00.168123 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47560323-ec74-43a5-9dc1-7a211cd9da25 admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:00.168673 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-63f774e6-e76b-4624-b6cb-ea59b44327b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:21:00.168785 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47560323-ec74-43a5-9dc1-7a211cd9da25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:00.173273 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47560323-ec74-43a5-9dc1-7a211cd9da25 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:00.256517 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f774e6-e76b-4624-b6cb-ea59b44327b7 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:00.282477 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1095/2172] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:21:00 2022] POST /identity/v3/users => generated 284 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:21:00.290908 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47560323-ec74-43a5-9dc1-7a211cd9da25 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:00.303058 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47560323-ec74-43a5-9dc1-7a211cd9da25 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:00.310675 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61949096-f8e1-481b-8097-0c7fab1e4056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:00.313474 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61949096-f8e1-481b-8097-0c7fab1e4056 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:00.313474 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61949096-f8e1-481b-8097-0c7fab1e4056 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:00.313474 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61949096-f8e1-481b-8097-0c7fab1e4056 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:00.313474 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61949096-f8e1-481b-8097-0c7fab1e4056 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:00.315142 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61949096-f8e1-481b-8097-0c7fab1e4056 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:00.332122 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1096/2173] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:21:00 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:21:00.337213 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61949096-f8e1-481b-8097-0c7fab1e4056 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:00.352852 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61949096-f8e1-481b-8097-0c7fab1e4056 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:00.358855 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a532431-5091-4734-b9c8-93ee35ee351e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:00.360762 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a532431-5091-4734-b9c8-93ee35ee351e admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:00.361153 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a532431-5091-4734-b9c8-93ee35ee351e admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:00.361718 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a532431-5091-4734-b9c8-93ee35ee351e admin admin] PATH_INFO: `/v3/projects/f48b2144ab374e649302fb49f7d22679/users/e8d16c536f38433abb28b440be8f7e92/roles/62f19ea272b447ccaab1576cd9bf4fd9` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:00.399877 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a532431-5091-4734-b9c8-93ee35ee351e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f48b2144ab374e649302fb49f7d22679, user_id=e8d16c536f38433abb28b440be8f7e92, role_id=62f19ea272b447ccaab1576cd9bf4fd9)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:00.414628 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a532431-5091-4734-b9c8-93ee35ee351e admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:00.416896 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a03e68-1a22-4763-95a5-1ee179fd75ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:00.417536 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a03e68-1a22-4763-95a5-1ee179fd75ef service neutron] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:00.417821 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a03e68-1a22-4763-95a5-1ee179fd75ef service neutron] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:00.418101 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a03e68-1a22-4763-95a5-1ee179fd75ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:00.465583 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a03e68-1a22-4763-95a5-1ee179fd75ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:00.467654 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a03e68-1a22-4763-95a5-1ee179fd75ef service neutron] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:00.478542 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:21:00.486787 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1078/2174] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:21:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 25 03:21:00.505076 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1097/2175] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:21:00 2022] PUT /identity/v3/projects/f48b2144ab374e649302fb49f7d22679/users/e8d16c536f38433abb28b440be8f7e92/roles/62f19ea272b447ccaab1576cd9bf4fd9 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:21:00.509576 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a03e68-1a22-4763-95a5-1ee179fd75ef service neutron] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:00.533924 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a03e68-1a22-4763-95a5-1ee179fd75ef service neutron] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:00.636769 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0bd8d3-5f0d-497d-8ac3-a5e9107cb342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:00.643020 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0bd8d3-5f0d-497d-8ac3-a5e9107cb342 admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:00.643020 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0bd8d3-5f0d-497d-8ac3-a5e9107cb342 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:00.643020 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0bd8d3-5f0d-497d-8ac3-a5e9107cb342 admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:00.643284 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0bd8d3-5f0d-497d-8ac3-a5e9107cb342 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:00.646684 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0bd8d3-5f0d-497d-8ac3-a5e9107cb342 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:00.681158 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1079/2176] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:21:00 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:21:00.681317 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a532431-5091-4734-b9c8-93ee35ee351e admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:00.693613 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a532431-5091-4734-b9c8-93ee35ee351e admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:00.707534 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd81f16c-85d4-4b36-9688-efc5cd7ae12a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:00.708847 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd81f16c-85d4-4b36-9688-efc5cd7ae12a admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:00.709203 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd81f16c-85d4-4b36-9688-efc5cd7ae12a admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:00.709530 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd81f16c-85d4-4b36-9688-efc5cd7ae12a admin admin] PATH_INFO: `/v3/projects/f48b2144ab374e649302fb49f7d22679/users/e8d16c536f38433abb28b440be8f7e92/roles/62f19ea272b447ccaab1576cd9bf4fd9` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:00.717680 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd81f16c-85d4-4b36-9688-efc5cd7ae12a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f48b2144ab374e649302fb49f7d22679, user_id=e8d16c536f38433abb28b440be8f7e92, role_id=62f19ea272b447ccaab1576cd9bf4fd9)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:00.720900 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd81f16c-85d4-4b36-9688-efc5cd7ae12a admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:00.747031 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1098/2177] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:21:00 2022] PUT /identity/v3/projects/f48b2144ab374e649302fb49f7d22679/users/e8d16c536f38433abb28b440be8f7e92/roles/62f19ea272b447ccaab1576cd9bf4fd9 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:21:00.765852 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34279510-400c-412f-92f1-8ccc5fde743f None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:00.766140 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34279510-400c-412f-92f1-8ccc5fde743f None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:00.766462 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34279510-400c-412f-92f1-8ccc5fde743f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:00.812642 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-34279510-400c-412f-92f1-8ccc5fde743f None None] MFA Rules not processed for user `e8d16c536f38433abb28b440be8f7e92`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:21:00.814491 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-34279510-400c-412f-92f1-8ccc5fde743f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:21:01.013645 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1080/2178] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:21:00 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:21:01.023792 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd81f16c-85d4-4b36-9688-efc5cd7ae12a admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:01.026158 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-fd81f16c-85d4-4b36-9688-efc5cd7ae12a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:21:01.119048 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd81f16c-85d4-4b36-9688-efc5cd7ae12a admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:01.125787 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004aa4a6-6e16-4ab1-9b07-c739a7876584 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60193ba9e1c14df3bc9eebcc6f28f2d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:01.126672 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004aa4a6-6e16-4ab1-9b07-c739a7876584 service cinder] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:01.131358 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004aa4a6-6e16-4ab1-9b07-c739a7876584 service cinder] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:01.131358 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004aa4a6-6e16-4ab1-9b07-c739a7876584 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:01.145147 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004aa4a6-6e16-4ab1-9b07-c739a7876584 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:01.146943 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004aa4a6-6e16-4ab1-9b07-c739a7876584 service cinder] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:01.160195 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1099/2179] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:21:01 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:21:01.448353 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34279510-400c-412f-92f1-8ccc5fde743f None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:01.462535 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34279510-400c-412f-92f1-8ccc5fde743f None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:01.466211 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc44ac22-ae47-491b-8a48-1e386affd08b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60193ba9e1c14df3bc9eebcc6f28f2d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:01.466899 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc44ac22-ae47-491b-8a48-1e386affd08b service cinder] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:01.467273 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc44ac22-ae47-491b-8a48-1e386affd08b service cinder] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:01.467481 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc44ac22-ae47-491b-8a48-1e386affd08b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:01.469490 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-cc44ac22-ae47-491b-8a48-1e386affd08b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:21:01.580989 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc44ac22-ae47-491b-8a48-1e386affd08b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:01.582525 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc44ac22-ae47-491b-8a48-1e386affd08b service cinder] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:01.606398 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1081/2180] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:21:01 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:21:01.932150 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004aa4a6-6e16-4ab1-9b07-c739a7876584 service cinder] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:01.936141 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-004aa4a6-6e16-4ab1-9b07-c739a7876584 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:21:02.035081 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004aa4a6-6e16-4ab1-9b07-c739a7876584 service cinder] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:02.040419 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a94cc30-feb0-4825-b0a6-4bb5b6af59bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:02.041071 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a94cc30-feb0-4825-b0a6-4bb5b6af59bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:02.041347 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a94cc30-feb0-4825-b0a6-4bb5b6af59bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:02.043723 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a94cc30-feb0-4825-b0a6-4bb5b6af59bb admin admin] PATH_INFO: `/v3/projects/aade369816e0473e856734c4233a69d6` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:02.043723 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a94cc30-feb0-4825-b0a6-4bb5b6af59bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=aade369816e0473e856734c4233a69d6)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:02.044864 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a94cc30-feb0-4825-b0a6-4bb5b6af59bb admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:02.165724 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:21:02.169451 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-5a94cc30-feb0-4825-b0a6-4bb5b6af59bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aade369816e0473e856734c4233a69d6', 'request_id': 'req-5a94cc30-feb0-4825-b0a6-4bb5b6af59bb'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:02.219774 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-5a94cc30-feb0-4825-b0a6-4bb5b6af59bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aade369816e0473e856734c4233a69d6', 'request_id': 'req-5a94cc30-feb0-4825-b0a6-4bb5b6af59bb'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:02.234154 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-5a94cc30-feb0-4825-b0a6-4bb5b6af59bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aade369816e0473e856734c4233a69d6', 'request_id': 'req-5a94cc30-feb0-4825-b0a6-4bb5b6af59bb'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:02.248193 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-5a94cc30-feb0-4825-b0a6-4bb5b6af59bb admin admin] The token cache is being invalidate because project aade369816e0473e856734c4233a69d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:21:02.248473 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-5a94cc30-feb0-4825-b0a6-4bb5b6af59bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:02.249141 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-5a94cc30-feb0-4825-b0a6-4bb5b6af59bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:02.250707 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1100/2181] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:21:01 2022] DELETE /identity/v3/projects/aade369816e0473e856734c4233a69d6 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:21:02.255666 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc44ac22-ae47-491b-8a48-1e386affd08b service cinder] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:02.266715 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc44ac22-ae47-491b-8a48-1e386affd08b service cinder] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:02.285532 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:21:02.370036 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12c3cf2-440b-4bf0-81de-6bee4c4996c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:02.370872 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12c3cf2-440b-4bf0-81de-6bee4c4996c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:02.377373 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12c3cf2-440b-4bf0-81de-6bee4c4996c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:02.377373 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12c3cf2-440b-4bf0-81de-6bee4c4996c0 admin admin] PATH_INFO: `/v3/users/34701ff69166436bbdad110b841b772f` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:02.377373 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12c3cf2-440b-4bf0-81de-6bee4c4996c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34701ff69166436bbdad110b841b772f)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:02.380360 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12c3cf2-440b-4bf0-81de-6bee4c4996c0 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:02.685770 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-c12c3cf2-440b-4bf0-81de-6bee4c4996c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34701ff69166436bbdad110b841b772f'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:02.806129 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-c12c3cf2-440b-4bf0-81de-6bee4c4996c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34701ff69166436bbdad110b841b772f'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:02.806919 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-c12c3cf2-440b-4bf0-81de-6bee4c4996c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34701ff69166436bbdad110b841b772f'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:02.807611 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-c12c3cf2-440b-4bf0-81de-6bee4c4996c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34701ff69166436bbdad110b841b772f'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:02.831583 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-c12c3cf2-440b-4bf0-81de-6bee4c4996c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34701ff69166436bbdad110b841b772f'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:02.874326 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1082/2182] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:21:02 2022] DELETE /identity/v3/users/34701ff69166436bbdad110b841b772f => generated 0 bytes in 616 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:21:03.476334 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a94cc30-feb0-4825-b0a6-4bb5b6af59bb admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:03.478443 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-5a94cc30-feb0-4825-b0a6-4bb5b6af59bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:21:03.498190 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a94cc30-feb0-4825-b0a6-4bb5b6af59bb admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:03.539250 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12c3cf2-440b-4bf0-81de-6bee4c4996c0 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:03.540891 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-c12c3cf2-440b-4bf0-81de-6bee4c4996c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:21:03.568677 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf76f79-7821-42b7-8a5e-e930f9d57a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:03.569721 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf76f79-7821-42b7-8a5e-e930f9d57a43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:03.570190 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf76f79-7821-42b7-8a5e-e930f9d57a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:03.578948 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf76f79-7821-42b7-8a5e-e930f9d57a43 admin admin] PATH_INFO: `/v3/users/827bfe2d861146b3aa67cfe147dc16d3` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:03.580272 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf76f79-7821-42b7-8a5e-e930f9d57a43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=827bfe2d861146b3aa67cfe147dc16d3)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:03.582406 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf76f79-7821-42b7-8a5e-e930f9d57a43 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:03.623761 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12c3cf2-440b-4bf0-81de-6bee4c4996c0 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:03.624374 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958d00a4-8f1a-4831-baa4-35e6f1a0f371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:03.625038 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958d00a4-8f1a-4831-baa4-35e6f1a0f371 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:03.625321 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958d00a4-8f1a-4831-baa4-35e6f1a0f371 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:03.625580 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958d00a4-8f1a-4831-baa4-35e6f1a0f371 admin admin] PATH_INFO: `/v3/projects/720e3ad46a5c44a99b73599e5ea304c6` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:03.626811 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958d00a4-8f1a-4831-baa4-35e6f1a0f371 admin admin] RBAC: Authorizing `identity:delete_project(project_id=720e3ad46a5c44a99b73599e5ea304c6)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:03.628771 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958d00a4-8f1a-4831-baa4-35e6f1a0f371 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:03.792732 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-0bf76f79-7821-42b7-8a5e-e930f9d57a43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '827bfe2d861146b3aa67cfe147dc16d3'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:03.794952 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:21:03.805067 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-958d00a4-8f1a-4831-baa4-35e6f1a0f371 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '720e3ad46a5c44a99b73599e5ea304c6', 'request_id': 'req-958d00a4-8f1a-4831-baa4-35e6f1a0f371'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:03.915452 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-0bf76f79-7821-42b7-8a5e-e930f9d57a43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '827bfe2d861146b3aa67cfe147dc16d3'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:03.916123 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-0bf76f79-7821-42b7-8a5e-e930f9d57a43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '827bfe2d861146b3aa67cfe147dc16d3'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:03.924236 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-0bf76f79-7821-42b7-8a5e-e930f9d57a43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '827bfe2d861146b3aa67cfe147dc16d3'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:03.946240 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-0bf76f79-7821-42b7-8a5e-e930f9d57a43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '827bfe2d861146b3aa67cfe147dc16d3'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:03.953895 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1101/2183] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:21:03 2022] DELETE /identity/v3/users/827bfe2d861146b3aa67cfe147dc16d3 => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:21:04.012287 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-958d00a4-8f1a-4831-baa4-35e6f1a0f371 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '720e3ad46a5c44a99b73599e5ea304c6', 'request_id': 'req-958d00a4-8f1a-4831-baa4-35e6f1a0f371'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:04.019676 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-958d00a4-8f1a-4831-baa4-35e6f1a0f371 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '720e3ad46a5c44a99b73599e5ea304c6', 'request_id': 'req-958d00a4-8f1a-4831-baa4-35e6f1a0f371'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:04.040587 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-958d00a4-8f1a-4831-baa4-35e6f1a0f371 admin admin] The token cache is being invalidate because project 720e3ad46a5c44a99b73599e5ea304c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81995) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:21:04.040587 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-958d00a4-8f1a-4831-baa4-35e6f1a0f371 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:04.042663 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-958d00a4-8f1a-4831-baa4-35e6f1a0f371 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:04.050999 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1083/2184] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:21:03 2022] DELETE /identity/v3/projects/720e3ad46a5c44a99b73599e5ea304c6 => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:21:04.051134 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf76f79-7821-42b7-8a5e-e930f9d57a43 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:04.052699 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-0bf76f79-7821-42b7-8a5e-e930f9d57a43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:21:04.125652 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:21:04.152020 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf76f79-7821-42b7-8a5e-e930f9d57a43 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:04.158880 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65acbbd-1e04-4986-bf66-a9d49f71ccb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:04.175214 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65acbbd-1e04-4986-bf66-a9d49f71ccb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:04.175562 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65acbbd-1e04-4986-bf66-a9d49f71ccb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:04.175903 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65acbbd-1e04-4986-bf66-a9d49f71ccb1 admin admin] PATH_INFO: `/v3/users/9930f948523a4c3fb641da242c014fab` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:04.180915 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65acbbd-1e04-4986-bf66-a9d49f71ccb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9930f948523a4c3fb641da242c014fab)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:04.182994 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65acbbd-1e04-4986-bf66-a9d49f71ccb1 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:04.345130 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958d00a4-8f1a-4831-baa4-35e6f1a0f371 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:04.346670 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-958d00a4-8f1a-4831-baa4-35e6f1a0f371 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:21:04.395917 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-d65acbbd-1e04-4986-bf66-a9d49f71ccb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9930f948523a4c3fb641da242c014fab'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:04.428691 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958d00a4-8f1a-4831-baa4-35e6f1a0f371 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:04.433464 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc887a9-5665-4356-95f3-8a3466b4feec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:04.434098 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc887a9-5665-4356-95f3-8a3466b4feec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:04.434406 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc887a9-5665-4356-95f3-8a3466b4feec admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:04.434675 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc887a9-5665-4356-95f3-8a3466b4feec admin admin] PATH_INFO: `/v3/projects/a5e956906b484196945c8deb97beaf5d` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:04.435768 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc887a9-5665-4356-95f3-8a3466b4feec admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5e956906b484196945c8deb97beaf5d)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:04.437697 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc887a9-5665-4356-95f3-8a3466b4feec admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:04.445924 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-d65acbbd-1e04-4986-bf66-a9d49f71ccb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9930f948523a4c3fb641da242c014fab'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:04.447020 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-d65acbbd-1e04-4986-bf66-a9d49f71ccb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9930f948523a4c3fb641da242c014fab'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:04.456021 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-d65acbbd-1e04-4986-bf66-a9d49f71ccb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9930f948523a4c3fb641da242c014fab'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:04.469654 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-d65acbbd-1e04-4986-bf66-a9d49f71ccb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9930f948523a4c3fb641da242c014fab'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:04.477586 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1102/2185] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:21:04 2022] DELETE /identity/v3/users/9930f948523a4c3fb641da242c014fab => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:21:04.572498 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-2dc887a9-5665-4356-95f3-8a3466b4feec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5e956906b484196945c8deb97beaf5d', 'request_id': 'req-2dc887a9-5665-4356-95f3-8a3466b4feec'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:04.680675 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-2dc887a9-5665-4356-95f3-8a3466b4feec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5e956906b484196945c8deb97beaf5d', 'request_id': 'req-2dc887a9-5665-4356-95f3-8a3466b4feec'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:04.687329 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-2dc887a9-5665-4356-95f3-8a3466b4feec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5e956906b484196945c8deb97beaf5d', 'request_id': 'req-2dc887a9-5665-4356-95f3-8a3466b4feec'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:04.703683 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-2dc887a9-5665-4356-95f3-8a3466b4feec admin admin] The token cache is being invalidate because project a5e956906b484196945c8deb97beaf5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81995) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:21:04.704302 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-2dc887a9-5665-4356-95f3-8a3466b4feec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:04.705566 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-2dc887a9-5665-4356-95f3-8a3466b4feec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:04.708501 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1084/2186] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:21:04 2022] DELETE /identity/v3/projects/a5e956906b484196945c8deb97beaf5d => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:21:04.718729 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bf5e6f-7a1a-4912-b8b8-bb38e94c6648 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:04.719028 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bf5e6f-7a1a-4912-b8b8-bb38e94c6648 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:04.719377 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bf5e6f-7a1a-4912-b8b8-bb38e94c6648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:04.778093 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-b4bf5e6f-7a1a-4912-b8b8-bb38e94c6648 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:21:04.782254 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-b4bf5e6f-7a1a-4912-b8b8-bb38e94c6648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:21:04.862352 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1103/2187] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:21:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:21:04.870496 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4401a3f3-fdb3-46f3-845b-51a1b3b6d45a None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:04.873520 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4401a3f3-fdb3-46f3-845b-51a1b3b6d45a None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:04.873520 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4401a3f3-fdb3-46f3-845b-51a1b3b6d45a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:04.910809 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-4401a3f3-fdb3-46f3-845b-51a1b3b6d45a None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:21:04.913104 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-4401a3f3-fdb3-46f3-845b-51a1b3b6d45a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:21:04.932170 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1085/2188] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:21:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:21:04.933805 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bf5e6f-7a1a-4912-b8b8-bb38e94c6648 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:04.946314 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bf5e6f-7a1a-4912-b8b8-bb38e94c6648 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:04.952248 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9da4ca-d8c4-4517-a487-19debec06c6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:04.953709 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9da4ca-d8c4-4517-a487-19debec06c6c admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:04.953709 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9da4ca-d8c4-4517-a487-19debec06c6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:04.953709 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9da4ca-d8c4-4517-a487-19debec06c6c admin admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:04.954097 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9da4ca-d8c4-4517-a487-19debec06c6c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:21:04.954529 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9da4ca-d8c4-4517-a487-19debec06c6c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:04.956353 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9da4ca-d8c4-4517-a487-19debec06c6c admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:04.972474 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1104/2189] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Aug 25 03:21:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:21:04.977090 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4401a3f3-fdb3-46f3-845b-51a1b3b6d45a None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:04.987137 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4401a3f3-fdb3-46f3-845b-51a1b3b6d45a None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:04.992259 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d371ca-0735-48b0-9d79-08d7d69d2bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:04.992945 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d371ca-0735-48b0-9d79-08d7d69d2bf0 admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:04.993297 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d371ca-0735-48b0-9d79-08d7d69d2bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:04.993701 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d371ca-0735-48b0-9d79-08d7d69d2bf0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:04.994479 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d371ca-0735-48b0-9d79-08d7d69d2bf0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:04.996467 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d371ca-0735-48b0-9d79-08d7d69d2bf0 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:05.128457 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9da4ca-d8c4-4517-a487-19debec06c6c admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:05.130242 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-1d9da4ca-d8c4-4517-a487-19debec06c6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:21:05.136734 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1086/2190] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:21:04 2022] POST /identity/v3/projects => generated 383 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:21:05.141119 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d371ca-0735-48b0-9d79-08d7d69d2bf0 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:05.159547 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9da4ca-d8c4-4517-a487-19debec06c6c admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:05.227480 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d371ca-0735-48b0-9d79-08d7d69d2bf0 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:05.228441 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4270136e-b3d1-4d0d-a02b-4d99155e85ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:05.229181 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4270136e-b3d1-4d0d-a02b-4d99155e85ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:05.229466 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4270136e-b3d1-4d0d-a02b-4d99155e85ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:05.229777 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4270136e-b3d1-4d0d-a02b-4d99155e85ea admin admin] PATH_INFO: `/v3/projects/b4f24674ab694d7b87213435752487b4` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:05.231016 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4270136e-b3d1-4d0d-a02b-4d99155e85ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4f24674ab694d7b87213435752487b4)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:05.233167 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98857d6-7ec9-4f57-8491-47a7aa9ea0af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:05.233304 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4270136e-b3d1-4d0d-a02b-4d99155e85ea admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:05.233851 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98857d6-7ec9-4f57-8491-47a7aa9ea0af admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:05.234501 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98857d6-7ec9-4f57-8491-47a7aa9ea0af admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:05.235401 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98857d6-7ec9-4f57-8491-47a7aa9ea0af admin admin] PATH_INFO: `/v3/users` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:05.236042 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98857d6-7ec9-4f57-8491-47a7aa9ea0af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:05.237951 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98857d6-7ec9-4f57-8491-47a7aa9ea0af admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:05.334569 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:21:05.385421 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1087/2191] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:21:05 2022] POST /identity/v3/users => generated 292 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:21:05.390268 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98857d6-7ec9-4f57-8491-47a7aa9ea0af admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:05.399988 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-4270136e-b3d1-4d0d-a02b-4d99155e85ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4f24674ab694d7b87213435752487b4', 'request_id': 'req-4270136e-b3d1-4d0d-a02b-4d99155e85ea'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:05.404136 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98857d6-7ec9-4f57-8491-47a7aa9ea0af admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:05.472708 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd4e728-6243-45c7-97de-6491c0273559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:05.473486 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd4e728-6243-45c7-97de-6491c0273559 admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:05.473827 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd4e728-6243-45c7-97de-6491c0273559 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:05.474142 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd4e728-6243-45c7-97de-6491c0273559 admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:05.474841 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd4e728-6243-45c7-97de-6491c0273559 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:05.478308 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd4e728-6243-45c7-97de-6491c0273559 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:05.490415 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1088/2192] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:21:05 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:21:05.491768 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd4e728-6243-45c7-97de-6491c0273559 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:05.514742 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd4e728-6243-45c7-97de-6491c0273559 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:05.524743 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4149c0a-8c86-4c13-a876-f1388dcba022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:05.526124 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-4270136e-b3d1-4d0d-a02b-4d99155e85ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4f24674ab694d7b87213435752487b4', 'request_id': 'req-4270136e-b3d1-4d0d-a02b-4d99155e85ea'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:05.530346 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4149c0a-8c86-4c13-a876-f1388dcba022 admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:05.530641 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4149c0a-8c86-4c13-a876-f1388dcba022 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:05.530919 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4149c0a-8c86-4c13-a876-f1388dcba022 admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:05.532742 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4149c0a-8c86-4c13-a876-f1388dcba022 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:05.534411 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4149c0a-8c86-4c13-a876-f1388dcba022 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:05.537755 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-4270136e-b3d1-4d0d-a02b-4d99155e85ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4f24674ab694d7b87213435752487b4', 'request_id': 'req-4270136e-b3d1-4d0d-a02b-4d99155e85ea'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:05.560942 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-4270136e-b3d1-4d0d-a02b-4d99155e85ea admin admin] The token cache is being invalidate because project b4f24674ab694d7b87213435752487b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:21:05.561403 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-4270136e-b3d1-4d0d-a02b-4d99155e85ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:05.562244 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-4270136e-b3d1-4d0d-a02b-4d99155e85ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:05.564169 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1105/2193] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:21:05 2022] DELETE /identity/v3/projects/b4f24674ab694d7b87213435752487b4 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:21:05.582161 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1089/2194] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:21:05 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:21:05.587487 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4149c0a-8c86-4c13-a876-f1388dcba022 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:05.587887 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c09e10e-7a36-495f-8d10-0e0ed42330e6 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:05.588297 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c09e10e-7a36-495f-8d10-0e0ed42330e6 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:05.588801 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c09e10e-7a36-495f-8d10-0e0ed42330e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:05.592793 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-c4149c0a-8c86-4c13-a876-f1388dcba022 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:21:05.621085 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-7c09e10e-7a36-495f-8d10-0e0ed42330e6 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:21:05.623145 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-7c09e10e-7a36-495f-8d10-0e0ed42330e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:21:05.639544 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1106/2195] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:21:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:21:05.641389 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6830d5-293d-49ed-b797-8f1682b0f616 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:05.641389 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6830d5-293d-49ed-b797-8f1682b0f616 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:05.641555 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6830d5-293d-49ed-b797-8f1682b0f616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:05.649467 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4149c0a-8c86-4c13-a876-f1388dcba022 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:05.662940 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-ac6830d5-293d-49ed-b797-8f1682b0f616 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:21:05.664398 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-ac6830d5-293d-49ed-b797-8f1682b0f616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:21:05.673801 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63f8a9e-08e4-4ea9-a3a0-90570668e6a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:05.674517 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63f8a9e-08e4-4ea9-a3a0-90570668e6a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:05.674804 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63f8a9e-08e4-4ea9-a3a0-90570668e6a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:05.675087 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63f8a9e-08e4-4ea9-a3a0-90570668e6a6 admin admin] PATH_INFO: `/v3/projects/a48e4c9aca394fc1a3bb497c26f2a9f1/users/68b69b25b19a4addb9aff7fb7a710e56/roles/3255f77192524a16bc405173106871fc` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:05.681689 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1107/2196] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:21:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:21:05.683903 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6830d5-293d-49ed-b797-8f1682b0f616 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:05.708819 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6830d5-293d-49ed-b797-8f1682b0f616 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:05.711305 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:21:05.722910 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63f8a9e-08e4-4ea9-a3a0-90570668e6a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a48e4c9aca394fc1a3bb497c26f2a9f1, user_id=68b69b25b19a4addb9aff7fb7a710e56, role_id=3255f77192524a16bc405173106871fc)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:05.723429 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bf7226-502c-4b49-b39f-dc3d79acf41e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:05.724171 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bf7226-502c-4b49-b39f-dc3d79acf41e admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:05.726738 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bf7226-502c-4b49-b39f-dc3d79acf41e admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:05.727552 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bf7226-502c-4b49-b39f-dc3d79acf41e admin admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:05.729122 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bf7226-502c-4b49-b39f-dc3d79acf41e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:21:05.729858 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bf7226-502c-4b49-b39f-dc3d79acf41e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:05.730115 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63f8a9e-08e4-4ea9-a3a0-90570668e6a6 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:05.732890 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bf7226-502c-4b49-b39f-dc3d79acf41e admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:05.760058 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1108/2197] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Aug 25 03:21:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:21:05.774006 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bf7226-502c-4b49-b39f-dc3d79acf41e admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:05.784258 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bf7226-502c-4b49-b39f-dc3d79acf41e admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:05.789209 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499276ed-ed1d-4d25-9047-37112abc2bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:05.789973 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499276ed-ed1d-4d25-9047-37112abc2bce admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:05.790356 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499276ed-ed1d-4d25-9047-37112abc2bce admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:05.790702 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499276ed-ed1d-4d25-9047-37112abc2bce admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:05.791399 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499276ed-ed1d-4d25-9047-37112abc2bce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:05.793406 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499276ed-ed1d-4d25-9047-37112abc2bce admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:05.858499 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1090/2198] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:21:05 2022] PUT /identity/v3/projects/a48e4c9aca394fc1a3bb497c26f2a9f1/users/68b69b25b19a4addb9aff7fb7a710e56/roles/3255f77192524a16bc405173106871fc => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:21:05.866588 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1627ebd0-f613-43eb-9523-fca304ded92e None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:05.866985 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1627ebd0-f613-43eb-9523-fca304ded92e None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:05.867274 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1627ebd0-f613-43eb-9523-fca304ded92e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:05.895737 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1109/2199] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:21:05 2022] POST /identity/v3/projects => generated 371 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:21:05.897796 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499276ed-ed1d-4d25-9047-37112abc2bce admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:05.904955 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-1627ebd0-f613-43eb-9523-fca304ded92e None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:21:05.906558 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-1627ebd0-f613-43eb-9523-fca304ded92e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:21:05.993878 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1091/2200] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:21:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:21:06.002709 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499276ed-ed1d-4d25-9047-37112abc2bce admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:06.006923 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b30ce6-9b49-4f08-8202-31e993f64792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:06.007534 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b30ce6-9b49-4f08-8202-31e993f64792 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:06.007816 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b30ce6-9b49-4f08-8202-31e993f64792 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:06.008083 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b30ce6-9b49-4f08-8202-31e993f64792 admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:06.008798 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b30ce6-9b49-4f08-8202-31e993f64792 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:06.010198 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1627ebd0-f613-43eb-9523-fca304ded92e None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:06.011375 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b30ce6-9b49-4f08-8202-31e993f64792 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:06.011864 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-1627ebd0-f613-43eb-9523-fca304ded92e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:21:06.076864 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1627ebd0-f613-43eb-9523-fca304ded92e None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:06.082370 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91093b0f-3b78-4a2c-a631-ffff0f973f44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:06.082974 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91093b0f-3b78-4a2c-a631-ffff0f973f44 service neutron] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:06.083253 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91093b0f-3b78-4a2c-a631-ffff0f973f44 service neutron] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:06.083522 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91093b0f-3b78-4a2c-a631-ffff0f973f44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:06.090419 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1110/2201] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:21:05 2022] POST /identity/v3/users => generated 286 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:21:06.092135 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b30ce6-9b49-4f08-8202-31e993f64792 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:06.093140 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91093b0f-3b78-4a2c-a631-ffff0f973f44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:06.094597 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91093b0f-3b78-4a2c-a631-ffff0f973f44 service neutron] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:06.102792 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b30ce6-9b49-4f08-8202-31e993f64792 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:06.104742 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1092/2202] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:21:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:21:06.109834 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a65f210-28d5-48ed-88a0-e26e38e04650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:06.110481 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a65f210-28d5-48ed-88a0-e26e38e04650 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:06.110845 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a65f210-28d5-48ed-88a0-e26e38e04650 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:06.111017 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a65f210-28d5-48ed-88a0-e26e38e04650 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:06.111646 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a65f210-28d5-48ed-88a0-e26e38e04650 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:06.113248 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a65f210-28d5-48ed-88a0-e26e38e04650 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:06.124421 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1111/2203] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:21:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:21:06.128950 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91093b0f-3b78-4a2c-a631-ffff0f973f44 service neutron] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:06.138987 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91093b0f-3b78-4a2c-a631-ffff0f973f44 service neutron] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:06.144648 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd85fb2-4c83-4fcf-b4d7-bd287c13b4de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:06.147819 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd85fb2-4c83-4fcf-b4d7-bd287c13b4de admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:06.147819 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd85fb2-4c83-4fcf-b4d7-bd287c13b4de admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:06.147819 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd85fb2-4c83-4fcf-b4d7-bd287c13b4de admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:06.147819 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd85fb2-4c83-4fcf-b4d7-bd287c13b4de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:06.148230 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd85fb2-4c83-4fcf-b4d7-bd287c13b4de admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:06.163808 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1093/2204] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:21:06 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:21:06.164713 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a65f210-28d5-48ed-88a0-e26e38e04650 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:06.178040 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a65f210-28d5-48ed-88a0-e26e38e04650 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:06.183151 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9b680c-1604-4088-b75f-4c806f492e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:06.183847 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9b680c-1604-4088-b75f-4c806f492e4f admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:06.184128 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9b680c-1604-4088-b75f-4c806f492e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:06.184402 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9b680c-1604-4088-b75f-4c806f492e4f admin admin] PATH_INFO: `/v3/projects/7e671c3bce4a4875b48ec740d026d5af/users/2c484664e87245b5903d15e67e08ae15/roles/3255f77192524a16bc405173106871fc` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:06.201402 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9b680c-1604-4088-b75f-4c806f492e4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e671c3bce4a4875b48ec740d026d5af, user_id=2c484664e87245b5903d15e67e08ae15, role_id=3255f77192524a16bc405173106871fc)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:06.205130 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9b680c-1604-4088-b75f-4c806f492e4f admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:06.383543 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1112/2205] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:21:06 2022] PUT /identity/v3/projects/7e671c3bce4a4875b48ec740d026d5af/users/2c484664e87245b5903d15e67e08ae15/roles/3255f77192524a16bc405173106871fc => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:21:06.394429 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65c37e3-0957-42ac-9a73-88b5bb8f615f None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:06.394722 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65c37e3-0957-42ac-9a73-88b5bb8f615f None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:06.395002 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65c37e3-0957-42ac-9a73-88b5bb8f615f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:06.425542 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-a65c37e3-0957-42ac-9a73-88b5bb8f615f None None] MFA Rules not processed for user `2c484664e87245b5903d15e67e08ae15`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:21:06.426873 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-a65c37e3-0957-42ac-9a73-88b5bb8f615f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:21:06.533645 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:21:06.549329 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1094/2206] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:21:06 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:21:06.557349 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9b680c-1604-4088-b75f-4c806f492e4f admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:06.559176 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-1d9b680c-1604-4088-b75f-4c806f492e4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:21:06.626897 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9b680c-1604-4088-b75f-4c806f492e4f admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:06.639477 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b32b9d-94fd-435b-bdf6-e4fee6bdbf5f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f89089b390c4b6599e54000c0fa082a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:06.640149 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b32b9d-94fd-435b-bdf6-e4fee6bdbf5f service glance] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:06.640436 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b32b9d-94fd-435b-bdf6-e4fee6bdbf5f service glance] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:06.641114 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b32b9d-94fd-435b-bdf6-e4fee6bdbf5f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:06.652269 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:21:06.657182 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b32b9d-94fd-435b-bdf6-e4fee6bdbf5f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:06.657182 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b32b9d-94fd-435b-bdf6-e4fee6bdbf5f service glance] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:06.666859 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1113/2207] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:21:06 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:21:12.005198 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65c37e3-0957-42ac-9a73-88b5bb8f615f None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:12.012147 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-a65c37e3-0957-42ac-9a73-88b5bb8f615f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:21:12.082730 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65c37e3-0957-42ac-9a73-88b5bb8f615f None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:12.093881 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22268a4b-8043-49fd-bfa5-3d9aafb79d94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:12.095165 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22268a4b-8043-49fd-bfa5-3d9aafb79d94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:12.095506 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22268a4b-8043-49fd-bfa5-3d9aafb79d94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:12.095787 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22268a4b-8043-49fd-bfa5-3d9aafb79d94 admin admin] PATH_INFO: `/v3/users/44f3ded2c4a34ecabfbaaac7b71f7aae` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:12.097516 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22268a4b-8043-49fd-bfa5-3d9aafb79d94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=44f3ded2c4a34ecabfbaaac7b71f7aae)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:12.101053 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22268a4b-8043-49fd-bfa5-3d9aafb79d94 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:12.358982 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:21:12.364444 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-22268a4b-8043-49fd-bfa5-3d9aafb79d94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44f3ded2c4a34ecabfbaaac7b71f7aae'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:12.436046 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-22268a4b-8043-49fd-bfa5-3d9aafb79d94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44f3ded2c4a34ecabfbaaac7b71f7aae'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:12.437096 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-22268a4b-8043-49fd-bfa5-3d9aafb79d94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44f3ded2c4a34ecabfbaaac7b71f7aae'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:12.437912 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-22268a4b-8043-49fd-bfa5-3d9aafb79d94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44f3ded2c4a34ecabfbaaac7b71f7aae'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:12.480578 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-22268a4b-8043-49fd-bfa5-3d9aafb79d94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44f3ded2c4a34ecabfbaaac7b71f7aae'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:12.512585 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1095/2208] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:21:12 2022] DELETE /identity/v3/users/44f3ded2c4a34ecabfbaaac7b71f7aae => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:21:12.514362 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d812c66d-fd0d-47dc-aa5c-430a6567adf9 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:12.514889 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d812c66d-fd0d-47dc-aa5c-430a6567adf9 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:12.514889 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d812c66d-fd0d-47dc-aa5c-430a6567adf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:12.561860 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-d812c66d-fd0d-47dc-aa5c-430a6567adf9 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:21:12.567093 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-d812c66d-fd0d-47dc-aa5c-430a6567adf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:21:12.699989 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1114/2209] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:21:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:21:12.715614 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22268a4b-8043-49fd-bfa5-3d9aafb79d94 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:12.717779 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-22268a4b-8043-49fd-bfa5-3d9aafb79d94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:21:12.742646 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c85f7-3393-4bd3-9291-693c3518be8b None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:12.743106 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c85f7-3393-4bd3-9291-693c3518be8b None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:12.743516 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c85f7-3393-4bd3-9291-693c3518be8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:12.780956 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-3e9c85f7-3393-4bd3-9291-693c3518be8b None None] MFA Rules not processed for user `68b69b25b19a4addb9aff7fb7a710e56`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:21:12.782165 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-3e9c85f7-3393-4bd3-9291-693c3518be8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:21:12.840775 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22268a4b-8043-49fd-bfa5-3d9aafb79d94 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:12.864292 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca1ac33-d4b7-4e96-98c6-c98bfe8eef6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:12.865195 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca1ac33-d4b7-4e96-98c6-c98bfe8eef6e service neutron] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:12.866197 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca1ac33-d4b7-4e96-98c6-c98bfe8eef6e service neutron] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:12.866656 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca1ac33-d4b7-4e96-98c6-c98bfe8eef6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:12.874016 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1115/2210] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:21:12 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:21:12.894562 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9c85f7-3393-4bd3-9291-693c3518be8b None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:12.894667 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca1ac33-d4b7-4e96-98c6-c98bfe8eef6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:12.894797 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-3e9c85f7-3393-4bd3-9291-693c3518be8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:21:12.894893 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca1ac33-d4b7-4e96-98c6-c98bfe8eef6e service neutron] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:12.905963 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1096/2211] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:21:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:21:12.944771 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:21:13.013748 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9c85f7-3393-4bd3-9291-693c3518be8b None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:13.023388 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7ff8f3-ad65-465e-ae68-92d120fb7ef3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cd171bf4df94f8fab397e42835feeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:13.024088 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7ff8f3-ad65-465e-ae68-92d120fb7ef3 service nova] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:13.024697 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7ff8f3-ad65-465e-ae68-92d120fb7ef3 service nova] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:13.024980 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7ff8f3-ad65-465e-ae68-92d120fb7ef3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:13.049668 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7ff8f3-ad65-465e-ae68-92d120fb7ef3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:13.052199 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7ff8f3-ad65-465e-ae68-92d120fb7ef3 service nova] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:13.064559 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1116/2212] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:21:12 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:21:13.504603 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca1ac33-d4b7-4e96-98c6-c98bfe8eef6e service neutron] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:13.506147 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-7ca1ac33-d4b7-4e96-98c6-c98bfe8eef6e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:21:13.525504 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca1ac33-d4b7-4e96-98c6-c98bfe8eef6e service neutron] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:13.531711 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40284fbf-b096-431b-acb8-4d9d6978f23d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:13.532557 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40284fbf-b096-431b-acb8-4d9d6978f23d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:13.532852 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40284fbf-b096-431b-acb8-4d9d6978f23d admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:13.533127 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40284fbf-b096-431b-acb8-4d9d6978f23d admin admin] PATH_INFO: `/v3/projects/d9bd9bbd655a4a5ba9b4ce00b93ca03b` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:13.534598 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40284fbf-b096-431b-acb8-4d9d6978f23d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9bd9bbd655a4a5ba9b4ce00b93ca03b)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:13.537538 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40284fbf-b096-431b-acb8-4d9d6978f23d admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:13.671577 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-40284fbf-b096-431b-acb8-4d9d6978f23d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9bd9bbd655a4a5ba9b4ce00b93ca03b', 'request_id': 'req-40284fbf-b096-431b-acb8-4d9d6978f23d'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:13.755359 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-40284fbf-b096-431b-acb8-4d9d6978f23d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9bd9bbd655a4a5ba9b4ce00b93ca03b', 'request_id': 'req-40284fbf-b096-431b-acb8-4d9d6978f23d'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:13.769311 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-40284fbf-b096-431b-acb8-4d9d6978f23d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9bd9bbd655a4a5ba9b4ce00b93ca03b', 'request_id': 'req-40284fbf-b096-431b-acb8-4d9d6978f23d'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:13.783563 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-40284fbf-b096-431b-acb8-4d9d6978f23d admin admin] The token cache is being invalidate because project d9bd9bbd655a4a5ba9b4ce00b93ca03b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81995) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:21:13.784437 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-40284fbf-b096-431b-acb8-4d9d6978f23d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:13.785187 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-40284fbf-b096-431b-acb8-4d9d6978f23d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:13.788000 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1097/2213] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:21:13 2022] DELETE /identity/v3/projects/d9bd9bbd655a4a5ba9b4ce00b93ca03b => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:21:15.936527 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010b1ac3-493c-4849-bfa0-e553ceae6ad7 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:15.937069 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010b1ac3-493c-4849-bfa0-e553ceae6ad7 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:15.938303 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010b1ac3-493c-4849-bfa0-e553ceae6ad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:15.966931 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-010b1ac3-493c-4849-bfa0-e553ceae6ad7 None None] MFA Rules not processed for user `68b69b25b19a4addb9aff7fb7a710e56`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:21:15.968163 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-010b1ac3-493c-4849-bfa0-e553ceae6ad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:21:16.053395 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1117/2214] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:21:15 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:21:23.779505 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=66,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:21:26.041102 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=47 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:21:32.901963 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40284fbf-b096-431b-acb8-4d9d6978f23d admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:32.903498 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-40284fbf-b096-431b-acb8-4d9d6978f23d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:21:32.976402 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40284fbf-b096-431b-acb8-4d9d6978f23d admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:32.980999 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75af337c-20b4-40e1-93fe-e59fff4b1c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:32.981676 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75af337c-20b4-40e1-93fe-e59fff4b1c40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:32.981966 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75af337c-20b4-40e1-93fe-e59fff4b1c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:32.982221 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75af337c-20b4-40e1-93fe-e59fff4b1c40 admin admin] PATH_INFO: `/v3/users/38a68944c88540568508d7b38ecf2200` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:32.983350 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75af337c-20b4-40e1-93fe-e59fff4b1c40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38a68944c88540568508d7b38ecf2200)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:32.985273 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75af337c-20b4-40e1-93fe-e59fff4b1c40 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:33.173686 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-75af337c-20b4-40e1-93fe-e59fff4b1c40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38a68944c88540568508d7b38ecf2200'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:33.234768 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-75af337c-20b4-40e1-93fe-e59fff4b1c40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38a68944c88540568508d7b38ecf2200'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:33.235453 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-75af337c-20b4-40e1-93fe-e59fff4b1c40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38a68944c88540568508d7b38ecf2200'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:33.237265 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-75af337c-20b4-40e1-93fe-e59fff4b1c40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38a68944c88540568508d7b38ecf2200'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:33.251543 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-75af337c-20b4-40e1-93fe-e59fff4b1c40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38a68944c88540568508d7b38ecf2200'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:33.265338 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1098/2215] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:21:32 2022] DELETE /identity/v3/users/38a68944c88540568508d7b38ecf2200 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:21:33.269557 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22765b8-e49a-4362-ac71-be071ecdf944 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:33.270479 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22765b8-e49a-4362-ac71-be071ecdf944 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:33.270842 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22765b8-e49a-4362-ac71-be071ecdf944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:33.291191 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-c22765b8-e49a-4362-ac71-be071ecdf944 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:21:33.292403 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-c22765b8-e49a-4362-ac71-be071ecdf944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:21:33.365615 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1118/2216] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:21:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:21:33.376117 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75af337c-20b4-40e1-93fe-e59fff4b1c40 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:33.378012 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-75af337c-20b4-40e1-93fe-e59fff4b1c40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:21:33.448733 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75af337c-20b4-40e1-93fe-e59fff4b1c40 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:33.452857 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8e24a6-2c36-4e1e-8616-425a58c61e1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:33.453471 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8e24a6-2c36-4e1e-8616-425a58c61e1d service neutron] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:33.453752 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8e24a6-2c36-4e1e-8616-425a58c61e1d service neutron] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:33.454030 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8e24a6-2c36-4e1e-8616-425a58c61e1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:33.464293 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8e24a6-2c36-4e1e-8616-425a58c61e1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:33.468563 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8e24a6-2c36-4e1e-8616-425a58c61e1d service neutron] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:33.476442 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1099/2217] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:21:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:21:33.950869 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22765b8-e49a-4362-ac71-be071ecdf944 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:33.952422 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-c22765b8-e49a-4362-ac71-be071ecdf944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:21:33.975846 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22765b8-e49a-4362-ac71-be071ecdf944 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:33.981439 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6f2232-b43d-4cfa-a521-9675667e5d51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:33.982243 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6f2232-b43d-4cfa-a521-9675667e5d51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:33.982616 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6f2232-b43d-4cfa-a521-9675667e5d51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:33.982897 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6f2232-b43d-4cfa-a521-9675667e5d51 admin admin] PATH_INFO: `/v3/projects/b9c761d2837842d19432cd120d5a0d2c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:33.984078 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6f2232-b43d-4cfa-a521-9675667e5d51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9c761d2837842d19432cd120d5a0d2c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:33.986163 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6f2232-b43d-4cfa-a521-9675667e5d51 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:34.092176 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-8b6f2232-b43d-4cfa-a521-9675667e5d51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9c761d2837842d19432cd120d5a0d2c', 'request_id': 'req-8b6f2232-b43d-4cfa-a521-9675667e5d51'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:34.154095 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-8b6f2232-b43d-4cfa-a521-9675667e5d51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9c761d2837842d19432cd120d5a0d2c', 'request_id': 'req-8b6f2232-b43d-4cfa-a521-9675667e5d51'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:34.160725 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-8b6f2232-b43d-4cfa-a521-9675667e5d51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9c761d2837842d19432cd120d5a0d2c', 'request_id': 'req-8b6f2232-b43d-4cfa-a521-9675667e5d51'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:34.172013 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-8b6f2232-b43d-4cfa-a521-9675667e5d51 admin admin] The token cache is being invalidate because project b9c761d2837842d19432cd120d5a0d2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:21:34.172308 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-8b6f2232-b43d-4cfa-a521-9675667e5d51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:34.172965 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-8b6f2232-b43d-4cfa-a521-9675667e5d51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:34.174497 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1119/2218] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:21:33 2022] DELETE /identity/v3/projects/b9c761d2837842d19432cd120d5a0d2c => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:21:34.178653 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8e24a6-2c36-4e1e-8616-425a58c61e1d service neutron] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:34.188813 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8e24a6-2c36-4e1e-8616-425a58c61e1d service neutron] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:34.208403 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:21:34.246764 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc97e35f-9462-4752-a5b9-97d83ec0eb98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:34.247406 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc97e35f-9462-4752-a5b9-97d83ec0eb98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:34.247687 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc97e35f-9462-4752-a5b9-97d83ec0eb98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:34.248102 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc97e35f-9462-4752-a5b9-97d83ec0eb98 admin admin] PATH_INFO: `/v3/users/e8d16c536f38433abb28b440be8f7e92` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:34.249209 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc97e35f-9462-4752-a5b9-97d83ec0eb98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8d16c536f38433abb28b440be8f7e92)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:34.251173 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc97e35f-9462-4752-a5b9-97d83ec0eb98 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:34.408061 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-cc97e35f-9462-4752-a5b9-97d83ec0eb98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8d16c536f38433abb28b440be8f7e92'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:34.468232 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-cc97e35f-9462-4752-a5b9-97d83ec0eb98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8d16c536f38433abb28b440be8f7e92'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:34.468862 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-cc97e35f-9462-4752-a5b9-97d83ec0eb98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8d16c536f38433abb28b440be8f7e92'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:34.469483 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-cc97e35f-9462-4752-a5b9-97d83ec0eb98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8d16c536f38433abb28b440be8f7e92'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:34.482752 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-cc97e35f-9462-4752-a5b9-97d83ec0eb98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8d16c536f38433abb28b440be8f7e92'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:34.496237 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1100/2219] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:21:34 2022] DELETE /identity/v3/users/e8d16c536f38433abb28b440be8f7e92 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:21:34.926811 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6f2232-b43d-4cfa-a521-9675667e5d51 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:34.928528 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-8b6f2232-b43d-4cfa-a521-9675667e5d51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:21:34.944116 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6f2232-b43d-4cfa-a521-9675667e5d51 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:35.002160 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0d7d7f-bb6f-42f8-8ac7-d7c171eb1844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:35.002773 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0d7d7f-bb6f-42f8-8ac7-d7c171eb1844 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:35.003036 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0d7d7f-bb6f-42f8-8ac7-d7c171eb1844 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:35.003307 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0d7d7f-bb6f-42f8-8ac7-d7c171eb1844 admin admin] PATH_INFO: `/v3/projects/f48b2144ab374e649302fb49f7d22679` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:35.004442 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0d7d7f-bb6f-42f8-8ac7-d7c171eb1844 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f48b2144ab374e649302fb49f7d22679)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:35.006399 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0d7d7f-bb6f-42f8-8ac7-d7c171eb1844 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:35.133681 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:21:35.150685 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-7d0d7d7f-bb6f-42f8-8ac7-d7c171eb1844 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f48b2144ab374e649302fb49f7d22679', 'request_id': 'req-7d0d7d7f-bb6f-42f8-8ac7-d7c171eb1844'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:35.223216 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-7d0d7d7f-bb6f-42f8-8ac7-d7c171eb1844 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f48b2144ab374e649302fb49f7d22679', 'request_id': 'req-7d0d7d7f-bb6f-42f8-8ac7-d7c171eb1844'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:35.229793 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-7d0d7d7f-bb6f-42f8-8ac7-d7c171eb1844 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f48b2144ab374e649302fb49f7d22679', 'request_id': 'req-7d0d7d7f-bb6f-42f8-8ac7-d7c171eb1844'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:35.241968 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-7d0d7d7f-bb6f-42f8-8ac7-d7c171eb1844 admin admin] The token cache is being invalidate because project f48b2144ab374e649302fb49f7d22679 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:21:35.242287 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-7d0d7d7f-bb6f-42f8-8ac7-d7c171eb1844 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:35.242954 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-7d0d7d7f-bb6f-42f8-8ac7-d7c171eb1844 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:35.244526 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1120/2220] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:21:34 2022] DELETE /identity/v3/projects/f48b2144ab374e649302fb49f7d22679 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:21:35.249665 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cb1dc9-c11c-4216-b551-0e6940d1c43a None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:35.250144 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cb1dc9-c11c-4216-b551-0e6940d1c43a None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:35.250241 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cb1dc9-c11c-4216-b551-0e6940d1c43a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:35.269882 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-90cb1dc9-c11c-4216-b551-0e6940d1c43a None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:21:35.271051 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-90cb1dc9-c11c-4216-b551-0e6940d1c43a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:21:35.331375 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1101/2221] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:21:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:21:35.339937 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7e416a-ea49-4a0c-be1d-928c6e034511 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:35.340340 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7e416a-ea49-4a0c-be1d-928c6e034511 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:35.340622 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7e416a-ea49-4a0c-be1d-928c6e034511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:35.360326 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-0d7e416a-ea49-4a0c-be1d-928c6e034511 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:21:35.361533 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-0d7e416a-ea49-4a0c-be1d-928c6e034511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:21:35.371390 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1121/2222] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:21:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:21:35.375779 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cb1dc9-c11c-4216-b551-0e6940d1c43a None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:35.386256 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cb1dc9-c11c-4216-b551-0e6940d1c43a None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:35.391012 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e58ba5-9b0e-46c3-b80a-510d869a1084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:35.391621 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e58ba5-9b0e-46c3-b80a-510d869a1084 admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:35.391903 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e58ba5-9b0e-46c3-b80a-510d869a1084 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:35.392178 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e58ba5-9b0e-46c3-b80a-510d869a1084 admin admin] PATH_INFO: `/v3/domains` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:35.392838 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e58ba5-9b0e-46c3-b80a-510d869a1084 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:21:35.393145 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e58ba5-9b0e-46c3-b80a-510d869a1084 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:35.394774 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e58ba5-9b0e-46c3-b80a-510d869a1084 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:35.407196 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1102/2223] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Aug 25 03:21:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:21:35.411488 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7e416a-ea49-4a0c-be1d-928c6e034511 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:35.420422 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7e416a-ea49-4a0c-be1d-928c6e034511 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:35.425097 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df61485-bd98-4357-b8af-d2e671af1b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:35.425683 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df61485-bd98-4357-b8af-d2e671af1b9e admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:35.425955 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df61485-bd98-4357-b8af-d2e671af1b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:35.426224 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df61485-bd98-4357-b8af-d2e671af1b9e admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:35.426876 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df61485-bd98-4357-b8af-d2e671af1b9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:35.428749 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df61485-bd98-4357-b8af-d2e671af1b9e admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:35.509880 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1122/2224] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:21:35 2022] POST /identity/v3/projects => generated 369 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:21:35.514205 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e58ba5-9b0e-46c3-b80a-510d869a1084 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:35.525277 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e58ba5-9b0e-46c3-b80a-510d869a1084 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:35.551015 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:21:35.592276 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3fd16c-e39e-4fc5-b020-3f96db5b07d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:35.592935 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3fd16c-e39e-4fc5-b020-3f96db5b07d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:35.593219 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3fd16c-e39e-4fc5-b020-3f96db5b07d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:35.593497 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3fd16c-e39e-4fc5-b020-3f96db5b07d2 admin admin] PATH_INFO: `/v3/users` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:35.594127 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3fd16c-e39e-4fc5-b020-3f96db5b07d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:35.596076 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3fd16c-e39e-4fc5-b020-3f96db5b07d2 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:35.704279 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1103/2225] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:21:35 2022] POST /identity/v3/users => generated 285 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:21:35.712965 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df61485-bd98-4357-b8af-d2e671af1b9e admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:35.723064 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df61485-bd98-4357-b8af-d2e671af1b9e admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:35.727631 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548325b2-bc8b-46a7-91d7-68db999297e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:35.728256 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548325b2-bc8b-46a7-91d7-68db999297e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:35.728535 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548325b2-bc8b-46a7-91d7-68db999297e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:35.728804 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548325b2-bc8b-46a7-91d7-68db999297e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:35.729444 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548325b2-bc8b-46a7-91d7-68db999297e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:35.731059 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548325b2-bc8b-46a7-91d7-68db999297e7 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:35.740723 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1123/2226] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:21:35 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:21:35.745172 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3fd16c-e39e-4fc5-b020-3f96db5b07d2 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:35.755059 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3fd16c-e39e-4fc5-b020-3f96db5b07d2 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:35.760237 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e42ebc-a1ad-4cb3-8311-bc47aa2f1be9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:35.760827 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e42ebc-a1ad-4cb3-8311-bc47aa2f1be9 admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:35.761111 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e42ebc-a1ad-4cb3-8311-bc47aa2f1be9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:35.761388 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e42ebc-a1ad-4cb3-8311-bc47aa2f1be9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:35.762032 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e42ebc-a1ad-4cb3-8311-bc47aa2f1be9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:35.763664 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e42ebc-a1ad-4cb3-8311-bc47aa2f1be9 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:35.773350 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1104/2227] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:21:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:21:35.777763 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548325b2-bc8b-46a7-91d7-68db999297e7 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:35.787498 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548325b2-bc8b-46a7-91d7-68db999297e7 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:35.791987 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b24b7d-8294-4a0a-af30-b590d96445ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:35.792634 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b24b7d-8294-4a0a-af30-b590d96445ec admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:35.793128 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b24b7d-8294-4a0a-af30-b590d96445ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:35.793415 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b24b7d-8294-4a0a-af30-b590d96445ec admin admin] PATH_INFO: `/v3/projects/de196112800c41889b2c77f78436236e/users/813610134dfe470abd005d9aa6aadc5c/roles/3255f77192524a16bc405173106871fc` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:35.808317 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b24b7d-8294-4a0a-af30-b590d96445ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=de196112800c41889b2c77f78436236e, user_id=813610134dfe470abd005d9aa6aadc5c, role_id=3255f77192524a16bc405173106871fc)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:35.810990 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b24b7d-8294-4a0a-af30-b590d96445ec admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:35.891806 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1124/2228] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:21:35 2022] PUT /identity/v3/projects/de196112800c41889b2c77f78436236e/users/813610134dfe470abd005d9aa6aadc5c/roles/3255f77192524a16bc405173106871fc => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:21:35.905408 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2928439-fefa-442f-975d-45e7f5e10206 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:35.905901 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2928439-fefa-442f-975d-45e7f5e10206 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:35.906322 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2928439-fefa-442f-975d-45e7f5e10206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:35.936088 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-e2928439-fefa-442f-975d-45e7f5e10206 None None] MFA Rules not processed for user `813610134dfe470abd005d9aa6aadc5c`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:21:35.937380 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-e2928439-fefa-442f-975d-45e7f5e10206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:21:36.016726 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1105/2229] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:21:35 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:21:36.021191 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b24b7d-8294-4a0a-af30-b590d96445ec admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:36.082524 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b24b7d-8294-4a0a-af30-b590d96445ec admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:36.087137 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e33918-061b-49b4-bd3a-f8e8eee7caa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:36.087740 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e33918-061b-49b4-bd3a-f8e8eee7caa6 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:36.088024 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e33918-061b-49b4-bd3a-f8e8eee7caa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:36.088302 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e33918-061b-49b4-bd3a-f8e8eee7caa6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:36.088930 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e33918-061b-49b4-bd3a-f8e8eee7caa6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:36.090854 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e33918-061b-49b4-bd3a-f8e8eee7caa6 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:36.172260 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1125/2230] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:21:36 2022] POST /identity/v3/projects => generated 367 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:21:36.176267 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2928439-fefa-442f-975d-45e7f5e10206 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:36.187690 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2928439-fefa-442f-975d-45e7f5e10206 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:36.244500 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85c83a4-f64b-4bc6-a076-f4b7fd1499c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:36.245133 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c83a4-f64b-4bc6-a076-f4b7fd1499c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:36.245397 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c83a4-f64b-4bc6-a076-f4b7fd1499c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:36.245669 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c83a4-f64b-4bc6-a076-f4b7fd1499c7 admin admin] PATH_INFO: `/v3/users` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:36.246320 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85c83a4-f64b-4bc6-a076-f4b7fd1499c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:36.248231 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85c83a4-f64b-4bc6-a076-f4b7fd1499c7 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:36.335520 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1106/2231] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:21:36 2022] POST /identity/v3/users => generated 290 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:21:36.337306 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e33918-061b-49b4-bd3a-f8e8eee7caa6 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:36.347669 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e33918-061b-49b4-bd3a-f8e8eee7caa6 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:36.353478 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ad7917-8b35-4ca8-ab33-82948b9aa1e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:36.354111 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ad7917-8b35-4ca8-ab33-82948b9aa1e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:36.359299 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ad7917-8b35-4ca8-ab33-82948b9aa1e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:36.359299 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ad7917-8b35-4ca8-ab33-82948b9aa1e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:36.359299 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ad7917-8b35-4ca8-ab33-82948b9aa1e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:36.359299 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ad7917-8b35-4ca8-ab33-82948b9aa1e3 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:36.368463 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1126/2232] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:21:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:21:36.373188 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85c83a4-f64b-4bc6-a076-f4b7fd1499c7 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:36.383711 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85c83a4-f64b-4bc6-a076-f4b7fd1499c7 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:36.388676 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8561d6-cbcf-44e1-84ce-afb07eabc680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:36.389337 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8561d6-cbcf-44e1-84ce-afb07eabc680 admin admin] REQUEST_METHOD: `PUT` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:36.389618 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8561d6-cbcf-44e1-84ce-afb07eabc680 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:36.389894 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8561d6-cbcf-44e1-84ce-afb07eabc680 admin admin] PATH_INFO: `/v3/projects/83fbcf3a7e2c451ab1a8060ea26aa19b/users/dc94d749342741699df4d21831c38185/roles/62f19ea272b447ccaab1576cd9bf4fd9` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:36.408959 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8561d6-cbcf-44e1-84ce-afb07eabc680 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83fbcf3a7e2c451ab1a8060ea26aa19b, user_id=dc94d749342741699df4d21831c38185, role_id=62f19ea272b447ccaab1576cd9bf4fd9)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:36.411768 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8561d6-cbcf-44e1-84ce-afb07eabc680 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:36.474963 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1107/2233] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:21:36 2022] PUT /identity/v3/projects/83fbcf3a7e2c451ab1a8060ea26aa19b/users/dc94d749342741699df4d21831c38185/roles/62f19ea272b447ccaab1576cd9bf4fd9 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:21:36.479441 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ad7917-8b35-4ca8-ab33-82948b9aa1e3 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:36.490103 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ad7917-8b35-4ca8-ab33-82948b9aa1e3 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:36.512136 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:21:36.548215 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff43ea9b-e257-4261-89c0-175a50df184f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:36.548846 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff43ea9b-e257-4261-89c0-175a50df184f admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:36.549124 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff43ea9b-e257-4261-89c0-175a50df184f admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:36.549395 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff43ea9b-e257-4261-89c0-175a50df184f admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:36.550028 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff43ea9b-e257-4261-89c0-175a50df184f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:36.551657 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff43ea9b-e257-4261-89c0-175a50df184f admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:36.561862 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1127/2234] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:21:36 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:21:36.565912 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8561d6-cbcf-44e1-84ce-afb07eabc680 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:36.571501 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:21:36.577060 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8561d6-cbcf-44e1-84ce-afb07eabc680 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:36.581930 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0db92d-f37e-4855-b98d-0a5a00c228fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:36.582862 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0db92d-f37e-4855-b98d-0a5a00c228fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:36.583145 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0db92d-f37e-4855-b98d-0a5a00c228fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:36.583417 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0db92d-f37e-4855-b98d-0a5a00c228fd admin admin] PATH_INFO: `/v3/projects/83fbcf3a7e2c451ab1a8060ea26aa19b/users/dc94d749342741699df4d21831c38185/roles/62f19ea272b447ccaab1576cd9bf4fd9` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:36.586007 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0db92d-f37e-4855-b98d-0a5a00c228fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=83fbcf3a7e2c451ab1a8060ea26aa19b, user_id=dc94d749342741699df4d21831c38185, role_id=62f19ea272b447ccaab1576cd9bf4fd9)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:36.588941 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0db92d-f37e-4855-b98d-0a5a00c228fd admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:36.599422 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1108/2235] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:21:36 2022] PUT /identity/v3/projects/83fbcf3a7e2c451ab1a8060ea26aa19b/users/dc94d749342741699df4d21831c38185/roles/62f19ea272b447ccaab1576cd9bf4fd9 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:21:36.610214 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb00590-fc0d-4804-a002-c0fd025f9389 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:36.610558 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb00590-fc0d-4804-a002-c0fd025f9389 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:36.610843 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb00590-fc0d-4804-a002-c0fd025f9389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:36.639801 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-4eb00590-fc0d-4804-a002-c0fd025f9389 None None] MFA Rules not processed for user `dc94d749342741699df4d21831c38185`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:21:36.641012 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-4eb00590-fc0d-4804-a002-c0fd025f9389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:21:36.733437 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1128/2236] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:21:36 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:21:36.746855 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0db92d-f37e-4855-b98d-0a5a00c228fd admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:36.749148 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-3a0db92d-f37e-4855-b98d-0a5a00c228fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:21:36.809573 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0db92d-f37e-4855-b98d-0a5a00c228fd admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:36.813342 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6204a5b2-af7c-40d0-9aff-52842bbe4e1b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60193ba9e1c14df3bc9eebcc6f28f2d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:36.813955 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6204a5b2-af7c-40d0-9aff-52842bbe4e1b service cinder] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:36.814235 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6204a5b2-af7c-40d0-9aff-52842bbe4e1b service cinder] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:36.814538 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6204a5b2-af7c-40d0-9aff-52842bbe4e1b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:36.824578 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6204a5b2-af7c-40d0-9aff-52842bbe4e1b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:36.826048 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6204a5b2-af7c-40d0-9aff-52842bbe4e1b service cinder] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:36.836309 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1109/2237] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:21:36 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 25 03:21:36.934884 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb00590-fc0d-4804-a002-c0fd025f9389 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:36.936404 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-4eb00590-fc0d-4804-a002-c0fd025f9389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:21:36.948710 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb00590-fc0d-4804-a002-c0fd025f9389 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:36.952471 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b36129-a84d-434c-bb14-dc26cf87240d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60193ba9e1c14df3bc9eebcc6f28f2d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:36.953066 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b36129-a84d-434c-bb14-dc26cf87240d service cinder] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:36.953358 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b36129-a84d-434c-bb14-dc26cf87240d service cinder] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:36.953626 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b36129-a84d-434c-bb14-dc26cf87240d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:37.012230 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b36129-a84d-434c-bb14-dc26cf87240d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:37.013693 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b36129-a84d-434c-bb14-dc26cf87240d service cinder] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:37.023338 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1129/2238] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:21:36 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:21:42.047445 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6204a5b2-af7c-40d0-9aff-52842bbe4e1b service cinder] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:42.117414 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6204a5b2-af7c-40d0-9aff-52842bbe4e1b service cinder] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:42.123904 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6420b8c9-8167-41c0-888f-27b266fbf361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:42.123904 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6420b8c9-8167-41c0-888f-27b266fbf361 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:42.123904 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6420b8c9-8167-41c0-888f-27b266fbf361 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:42.124234 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6420b8c9-8167-41c0-888f-27b266fbf361 admin admin] PATH_INFO: `/v3/users/813610134dfe470abd005d9aa6aadc5c` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:42.125903 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6420b8c9-8167-41c0-888f-27b266fbf361 admin admin] RBAC: Authorizing `identity:delete_user(user_id=813610134dfe470abd005d9aa6aadc5c)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:42.128423 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6420b8c9-8167-41c0-888f-27b266fbf361 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:42.329107 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-6420b8c9-8167-41c0-888f-27b266fbf361 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '813610134dfe470abd005d9aa6aadc5c'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:42.399950 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-6420b8c9-8167-41c0-888f-27b266fbf361 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '813610134dfe470abd005d9aa6aadc5c'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:42.400784 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-6420b8c9-8167-41c0-888f-27b266fbf361 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '813610134dfe470abd005d9aa6aadc5c'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:42.401437 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-6420b8c9-8167-41c0-888f-27b266fbf361 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '813610134dfe470abd005d9aa6aadc5c'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:42.422571 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-6420b8c9-8167-41c0-888f-27b266fbf361 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '813610134dfe470abd005d9aa6aadc5c'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:42.440395 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1110/2239] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:21:42 2022] DELETE /identity/v3/users/813610134dfe470abd005d9aa6aadc5c => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:21:42.445943 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ae21fa-b737-4fa7-83ae-15ae3cafa2e9 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:42.446479 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ae21fa-b737-4fa7-83ae-15ae3cafa2e9 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:42.446581 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ae21fa-b737-4fa7-83ae-15ae3cafa2e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:42.468373 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-13ae21fa-b737-4fa7-83ae-15ae3cafa2e9 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:21:42.469721 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-13ae21fa-b737-4fa7-83ae-15ae3cafa2e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:21:42.516520 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:21:42.535064 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1130/2240] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:21:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:21:42.545700 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6420b8c9-8167-41c0-888f-27b266fbf361 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:42.547319 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-6420b8c9-8167-41c0-888f-27b266fbf361 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:21:42.560564 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:21:42.607611 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6420b8c9-8167-41c0-888f-27b266fbf361 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:42.611262 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5400904-eba7-49d0-aee2-d0ac2b2cab85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:42.611938 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5400904-eba7-49d0-aee2-d0ac2b2cab85 service neutron] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:42.612279 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5400904-eba7-49d0-aee2-d0ac2b2cab85 service neutron] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:42.612623 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5400904-eba7-49d0-aee2-d0ac2b2cab85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:42.622063 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5400904-eba7-49d0-aee2-d0ac2b2cab85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:42.623537 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5400904-eba7-49d0-aee2-d0ac2b2cab85 service neutron] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:42.634225 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1111/2241] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:21:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:21:43.123414 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ae21fa-b737-4fa7-83ae-15ae3cafa2e9 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:43.125963 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-13ae21fa-b737-4fa7-83ae-15ae3cafa2e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:21:43.157321 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ae21fa-b737-4fa7-83ae-15ae3cafa2e9 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:43.164015 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d00022-1e03-4bbb-85c5-dc347e25b2d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:43.164703 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d00022-1e03-4bbb-85c5-dc347e25b2d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:43.165300 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d00022-1e03-4bbb-85c5-dc347e25b2d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:43.165592 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d00022-1e03-4bbb-85c5-dc347e25b2d5 admin admin] PATH_INFO: `/v3/projects/de196112800c41889b2c77f78436236e` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:43.166904 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d00022-1e03-4bbb-85c5-dc347e25b2d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de196112800c41889b2c77f78436236e)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:43.168846 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d00022-1e03-4bbb-85c5-dc347e25b2d5 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:43.305216 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-e5d00022-1e03-4bbb-85c5-dc347e25b2d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de196112800c41889b2c77f78436236e', 'request_id': 'req-e5d00022-1e03-4bbb-85c5-dc347e25b2d5'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:43.387762 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-e5d00022-1e03-4bbb-85c5-dc347e25b2d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de196112800c41889b2c77f78436236e', 'request_id': 'req-e5d00022-1e03-4bbb-85c5-dc347e25b2d5'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:43.395768 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-e5d00022-1e03-4bbb-85c5-dc347e25b2d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de196112800c41889b2c77f78436236e', 'request_id': 'req-e5d00022-1e03-4bbb-85c5-dc347e25b2d5'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:43.408223 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-e5d00022-1e03-4bbb-85c5-dc347e25b2d5 admin admin] The token cache is being invalidate because project de196112800c41889b2c77f78436236e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:21:43.408449 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-e5d00022-1e03-4bbb-85c5-dc347e25b2d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:43.409103 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-e5d00022-1e03-4bbb-85c5-dc347e25b2d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:43.410678 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1131/2242] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:21:43 2022] DELETE /identity/v3/projects/de196112800c41889b2c77f78436236e => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:21:43.415243 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5400904-eba7-49d0-aee2-d0ac2b2cab85 service neutron] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:43.424815 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5400904-eba7-49d0-aee2-d0ac2b2cab85 service neutron] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:43.496777 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85d87f7-8517-4814-8318-4844d08e2faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:43.496777 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85d87f7-8517-4814-8318-4844d08e2faa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:43.496777 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85d87f7-8517-4814-8318-4844d08e2faa admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:43.496777 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85d87f7-8517-4814-8318-4844d08e2faa admin admin] PATH_INFO: `/v3/users/dc94d749342741699df4d21831c38185` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:43.497145 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85d87f7-8517-4814-8318-4844d08e2faa admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc94d749342741699df4d21831c38185)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:43.497816 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85d87f7-8517-4814-8318-4844d08e2faa admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:43.768499 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-c85d87f7-8517-4814-8318-4844d08e2faa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc94d749342741699df4d21831c38185'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:43.844958 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-c85d87f7-8517-4814-8318-4844d08e2faa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc94d749342741699df4d21831c38185'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:43.845629 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-c85d87f7-8517-4814-8318-4844d08e2faa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc94d749342741699df4d21831c38185'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:43.846243 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-c85d87f7-8517-4814-8318-4844d08e2faa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc94d749342741699df4d21831c38185'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:43.859872 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-c85d87f7-8517-4814-8318-4844d08e2faa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc94d749342741699df4d21831c38185'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:43.872963 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1112/2243] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:21:43 2022] DELETE /identity/v3/users/dc94d749342741699df4d21831c38185 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:21:44.277293 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d00022-1e03-4bbb-85c5-dc347e25b2d5 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:44.278840 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-e5d00022-1e03-4bbb-85c5-dc347e25b2d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:21:44.291844 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d00022-1e03-4bbb-85c5-dc347e25b2d5 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:44.353069 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69e2cae-105e-4f07-95f2-d5c20a9acb35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:44.353677 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69e2cae-105e-4f07-95f2-d5c20a9acb35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:44.353952 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69e2cae-105e-4f07-95f2-d5c20a9acb35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:44.354220 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69e2cae-105e-4f07-95f2-d5c20a9acb35 admin admin] PATH_INFO: `/v3/projects/83fbcf3a7e2c451ab1a8060ea26aa19b` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:44.355345 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69e2cae-105e-4f07-95f2-d5c20a9acb35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=83fbcf3a7e2c451ab1a8060ea26aa19b)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:44.357420 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69e2cae-105e-4f07-95f2-d5c20a9acb35 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:44.566351 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-a69e2cae-105e-4f07-95f2-d5c20a9acb35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83fbcf3a7e2c451ab1a8060ea26aa19b', 'request_id': 'req-a69e2cae-105e-4f07-95f2-d5c20a9acb35'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:44.634057 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-a69e2cae-105e-4f07-95f2-d5c20a9acb35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83fbcf3a7e2c451ab1a8060ea26aa19b', 'request_id': 'req-a69e2cae-105e-4f07-95f2-d5c20a9acb35'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:44.643826 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-a69e2cae-105e-4f07-95f2-d5c20a9acb35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83fbcf3a7e2c451ab1a8060ea26aa19b', 'request_id': 'req-a69e2cae-105e-4f07-95f2-d5c20a9acb35'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:44.651906 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:21:44.657107 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-a69e2cae-105e-4f07-95f2-d5c20a9acb35 admin admin] The token cache is being invalidate because project 83fbcf3a7e2c451ab1a8060ea26aa19b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:21:44.657398 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-a69e2cae-105e-4f07-95f2-d5c20a9acb35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:44.658054 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-a69e2cae-105e-4f07-95f2-d5c20a9acb35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:44.659576 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1132/2244] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:21:44 2022] DELETE /identity/v3/projects/83fbcf3a7e2c451ab1a8060ea26aa19b => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:21:44.668061 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce883a7-0213-419b-b9c7-585ed1afcb82 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:44.668712 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce883a7-0213-419b-b9c7-585ed1afcb82 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:44.669387 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce883a7-0213-419b-b9c7-585ed1afcb82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:44.691065 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-cce883a7-0213-419b-b9c7-585ed1afcb82 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:21:44.692296 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-cce883a7-0213-419b-b9c7-585ed1afcb82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:21:44.740033 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:21:44.787231 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1113/2245] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:21:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:21:44.798082 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ce6bdf-68d3-4b34-93c3-a22df701394d None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:44.798787 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ce6bdf-68d3-4b34-93c3-a22df701394d None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:44.804929 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ce6bdf-68d3-4b34-93c3-a22df701394d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:44.824892 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-f0ce6bdf-68d3-4b34-93c3-a22df701394d None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:21:44.826119 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-f0ce6bdf-68d3-4b34-93c3-a22df701394d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:21:44.834460 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1133/2246] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:21:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:21:44.838836 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce883a7-0213-419b-b9c7-585ed1afcb82 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:44.849985 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce883a7-0213-419b-b9c7-585ed1afcb82 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:44.854863 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d33d2d3-f7f4-41dc-aadb-c7a6094f68f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:44.856613 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d33d2d3-f7f4-41dc-aadb-c7a6094f68f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:44.856613 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d33d2d3-f7f4-41dc-aadb-c7a6094f68f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:44.856613 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d33d2d3-f7f4-41dc-aadb-c7a6094f68f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:44.856613 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d33d2d3-f7f4-41dc-aadb-c7a6094f68f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:21:44.856946 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d33d2d3-f7f4-41dc-aadb-c7a6094f68f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:44.858638 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d33d2d3-f7f4-41dc-aadb-c7a6094f68f2 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:44.871157 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1114/2247] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Aug 25 03:21:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:21:44.875687 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ce6bdf-68d3-4b34-93c3-a22df701394d None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:44.885968 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ce6bdf-68d3-4b34-93c3-a22df701394d None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:44.891512 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cf79e6-5814-41ec-885c-4ff0cc1da63e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:44.892164 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cf79e6-5814-41ec-885c-4ff0cc1da63e admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:44.892454 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cf79e6-5814-41ec-885c-4ff0cc1da63e admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:44.892715 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cf79e6-5814-41ec-885c-4ff0cc1da63e admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:44.893351 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cf79e6-5814-41ec-885c-4ff0cc1da63e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:44.895280 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cf79e6-5814-41ec-885c-4ff0cc1da63e admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:45.011646 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1134/2248] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:21:44 2022] POST /identity/v3/projects => generated 381 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:21:45.018329 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d33d2d3-f7f4-41dc-aadb-c7a6094f68f2 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:45.028832 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d33d2d3-f7f4-41dc-aadb-c7a6094f68f2 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:45.090862 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09caf360-704d-40df-98d7-0376b5cb411d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:45.091387 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09caf360-704d-40df-98d7-0376b5cb411d admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:45.091665 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09caf360-704d-40df-98d7-0376b5cb411d admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:45.091936 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09caf360-704d-40df-98d7-0376b5cb411d admin admin] PATH_INFO: `/v3/users` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:45.092735 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09caf360-704d-40df-98d7-0376b5cb411d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:45.095095 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09caf360-704d-40df-98d7-0376b5cb411d admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:45.202753 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1115/2249] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:21:45 2022] POST /identity/v3/users => generated 291 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:21:45.204277 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cf79e6-5814-41ec-885c-4ff0cc1da63e admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:45.214854 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cf79e6-5814-41ec-885c-4ff0cc1da63e admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:45.219791 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66ef5fa-1873-494b-8299-dce78e051861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:45.220408 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66ef5fa-1873-494b-8299-dce78e051861 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:45.220739 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66ef5fa-1873-494b-8299-dce78e051861 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:45.221014 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66ef5fa-1873-494b-8299-dce78e051861 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:45.221670 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66ef5fa-1873-494b-8299-dce78e051861 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:45.223296 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66ef5fa-1873-494b-8299-dce78e051861 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:45.237443 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1135/2250] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:21:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:21:45.238943 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09caf360-704d-40df-98d7-0376b5cb411d admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:45.250043 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09caf360-704d-40df-98d7-0376b5cb411d admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:45.254916 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c29dc6-f3a4-4e74-9e62-7316c2a16f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:45.255534 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c29dc6-f3a4-4e74-9e62-7316c2a16f07 admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:45.255814 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c29dc6-f3a4-4e74-9e62-7316c2a16f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:45.256091 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c29dc6-f3a4-4e74-9e62-7316c2a16f07 admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:45.256794 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c29dc6-f3a4-4e74-9e62-7316c2a16f07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:45.258649 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c29dc6-f3a4-4e74-9e62-7316c2a16f07 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:45.270317 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1116/2251] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:21:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 25 03:21:45.274510 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66ef5fa-1873-494b-8299-dce78e051861 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:45.285427 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66ef5fa-1873-494b-8299-dce78e051861 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:45.290780 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee58cd14-108c-473b-b1de-59587d826fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:45.291444 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee58cd14-108c-473b-b1de-59587d826fc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:45.291723 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee58cd14-108c-473b-b1de-59587d826fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:45.292432 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee58cd14-108c-473b-b1de-59587d826fc8 admin admin] PATH_INFO: `/v3/projects/4ae2b7c2698146cc83a6378704734a13/users/8490068a3789495783b1045e1ab12342/roles/3255f77192524a16bc405173106871fc` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:45.308331 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee58cd14-108c-473b-b1de-59587d826fc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ae2b7c2698146cc83a6378704734a13, user_id=8490068a3789495783b1045e1ab12342, role_id=3255f77192524a16bc405173106871fc)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:45.311523 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee58cd14-108c-473b-b1de-59587d826fc8 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:45.388780 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1136/2252] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:21:45 2022] PUT /identity/v3/projects/4ae2b7c2698146cc83a6378704734a13/users/8490068a3789495783b1045e1ab12342/roles/3255f77192524a16bc405173106871fc => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:21:45.400866 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b90af69-ba56-4f47-80ac-501e9f70f38e None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:45.401161 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b90af69-ba56-4f47-80ac-501e9f70f38e None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:45.401443 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b90af69-ba56-4f47-80ac-501e9f70f38e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:45.433282 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-3b90af69-ba56-4f47-80ac-501e9f70f38e None None] MFA Rules not processed for user `8490068a3789495783b1045e1ab12342`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:21:45.435665 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-3b90af69-ba56-4f47-80ac-501e9f70f38e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:21:45.510682 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1117/2253] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:21:45 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:21:45.518178 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee58cd14-108c-473b-b1de-59587d826fc8 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:45.615726 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee58cd14-108c-473b-b1de-59587d826fc8 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:45.623537 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad9d6ee-3cab-414e-b5b9-3f706e6fd024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:45.624614 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad9d6ee-3cab-414e-b5b9-3f706e6fd024 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:45.624614 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad9d6ee-3cab-414e-b5b9-3f706e6fd024 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:45.624820 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad9d6ee-3cab-414e-b5b9-3f706e6fd024 admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:45.625577 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad9d6ee-3cab-414e-b5b9-3f706e6fd024 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:45.627564 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad9d6ee-3cab-414e-b5b9-3f706e6fd024 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:45.734180 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1137/2254] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:21:45 2022] POST /identity/v3/projects => generated 383 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:21:45.740601 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b90af69-ba56-4f47-80ac-501e9f70f38e None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:45.754339 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b90af69-ba56-4f47-80ac-501e9f70f38e None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:45.787076 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:21:45.823611 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d63b7a-4d1d-4bf1-afce-4c38526d7cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:45.824496 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d63b7a-4d1d-4bf1-afce-4c38526d7cd0 admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:45.824775 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d63b7a-4d1d-4bf1-afce-4c38526d7cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:45.825284 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d63b7a-4d1d-4bf1-afce-4c38526d7cd0 admin admin] PATH_INFO: `/v3/users` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:45.825938 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d63b7a-4d1d-4bf1-afce-4c38526d7cd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:45.828334 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d63b7a-4d1d-4bf1-afce-4c38526d7cd0 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:45.921357 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1118/2255] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:21:45 2022] POST /identity/v3/users => generated 298 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:21:45.924117 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad9d6ee-3cab-414e-b5b9-3f706e6fd024 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:45.935022 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad9d6ee-3cab-414e-b5b9-3f706e6fd024 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:45.940055 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8757fe-72d5-42eb-a0b0-d5b23dfc542e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:45.940674 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8757fe-72d5-42eb-a0b0-d5b23dfc542e admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:45.941412 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8757fe-72d5-42eb-a0b0-d5b23dfc542e admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:45.941751 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8757fe-72d5-42eb-a0b0-d5b23dfc542e admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:45.942503 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8757fe-72d5-42eb-a0b0-d5b23dfc542e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:45.946011 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8757fe-72d5-42eb-a0b0-d5b23dfc542e admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:45.978107 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1138/2256] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:21:45 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:21:45.979593 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d63b7a-4d1d-4bf1-afce-4c38526d7cd0 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:45.996018 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d63b7a-4d1d-4bf1-afce-4c38526d7cd0 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:46.002875 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdccee8-67cd-4678-83af-65cb32808804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:46.003878 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdccee8-67cd-4678-83af-65cb32808804 admin admin] REQUEST_METHOD: `PUT` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:46.004158 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdccee8-67cd-4678-83af-65cb32808804 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:46.004434 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdccee8-67cd-4678-83af-65cb32808804 admin admin] PATH_INFO: `/v3/projects/c48b517b40e945a1aea2a25d53be49ce/users/dc986da9b803458fa1f77504c3822aeb/roles/62f19ea272b447ccaab1576cd9bf4fd9` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:46.038025 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdccee8-67cd-4678-83af-65cb32808804 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c48b517b40e945a1aea2a25d53be49ce, user_id=dc986da9b803458fa1f77504c3822aeb, role_id=62f19ea272b447ccaab1576cd9bf4fd9)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:46.047273 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdccee8-67cd-4678-83af-65cb32808804 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:46.049672 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8757fe-72d5-42eb-a0b0-d5b23dfc542e admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:46.051564 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-4a8757fe-72d5-42eb-a0b0-d5b23dfc542e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:21:46.068547 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8757fe-72d5-42eb-a0b0-d5b23dfc542e admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:46.078736 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e26580-a97b-430d-94ec-7241c9face31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:46.079468 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e26580-a97b-430d-94ec-7241c9face31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:46.080685 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e26580-a97b-430d-94ec-7241c9face31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:46.081045 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e26580-a97b-430d-94ec-7241c9face31 admin admin] PATH_INFO: `/v3/users/dfbf85b9e5b140ceba1fe0510ee1109f` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:46.091327 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e26580-a97b-430d-94ec-7241c9face31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dfbf85b9e5b140ceba1fe0510ee1109f)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:46.091327 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e26580-a97b-430d-94ec-7241c9face31 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:46.170777 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1119/2257] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:21:45 2022] PUT /identity/v3/projects/c48b517b40e945a1aea2a25d53be49ce/users/dc986da9b803458fa1f77504c3822aeb/roles/62f19ea272b447ccaab1576cd9bf4fd9 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:21:46.177307 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdccee8-67cd-4678-83af-65cb32808804 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:46.246601 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdccee8-67cd-4678-83af-65cb32808804 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:46.251808 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888b8e72-cdc4-46ea-b3d3-3897422f34e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:46.252545 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888b8e72-cdc4-46ea-b3d3-3897422f34e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:46.252893 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888b8e72-cdc4-46ea-b3d3-3897422f34e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:46.253232 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888b8e72-cdc4-46ea-b3d3-3897422f34e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:46.253969 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888b8e72-cdc4-46ea-b3d3-3897422f34e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:46.255740 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888b8e72-cdc4-46ea-b3d3-3897422f34e8 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:46.269617 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1120/2258] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:21:46 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 25 03:21:46.277852 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888b8e72-cdc4-46ea-b3d3-3897422f34e8 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:46.286599 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-69e26580-a97b-430d-94ec-7241c9face31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dfbf85b9e5b140ceba1fe0510ee1109f'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:46.302574 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888b8e72-cdc4-46ea-b3d3-3897422f34e8 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:46.314961 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59796f91-5a56-4f2b-a47e-f9a453496b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:46.315641 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59796f91-5a56-4f2b-a47e-f9a453496b06 admin admin] REQUEST_METHOD: `PUT` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:46.315920 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59796f91-5a56-4f2b-a47e-f9a453496b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:46.316211 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59796f91-5a56-4f2b-a47e-f9a453496b06 admin admin] PATH_INFO: `/v3/projects/c48b517b40e945a1aea2a25d53be49ce/users/dc986da9b803458fa1f77504c3822aeb/roles/62f19ea272b447ccaab1576cd9bf4fd9` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:46.318126 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59796f91-5a56-4f2b-a47e-f9a453496b06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c48b517b40e945a1aea2a25d53be49ce, user_id=dc986da9b803458fa1f77504c3822aeb, role_id=62f19ea272b447ccaab1576cd9bf4fd9)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:46.321557 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59796f91-5a56-4f2b-a47e-f9a453496b06 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:46.334472 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1121/2259] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:21:46 2022] PUT /identity/v3/projects/c48b517b40e945a1aea2a25d53be49ce/users/dc986da9b803458fa1f77504c3822aeb/roles/62f19ea272b447ccaab1576cd9bf4fd9 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:21:46.345022 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d140bc85-13fb-4dd0-a8b1-fdf56ccac1a6 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:46.345317 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d140bc85-13fb-4dd0-a8b1-fdf56ccac1a6 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:46.345600 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d140bc85-13fb-4dd0-a8b1-fdf56ccac1a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:46.371110 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-69e26580-a97b-430d-94ec-7241c9face31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dfbf85b9e5b140ceba1fe0510ee1109f'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:46.371828 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-69e26580-a97b-430d-94ec-7241c9face31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dfbf85b9e5b140ceba1fe0510ee1109f'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:46.379975 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-69e26580-a97b-430d-94ec-7241c9face31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dfbf85b9e5b140ceba1fe0510ee1109f'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:46.382701 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:21:46.383690 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-d140bc85-13fb-4dd0-a8b1-fdf56ccac1a6 None None] MFA Rules not processed for user `dc986da9b803458fa1f77504c3822aeb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:21:46.385131 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-d140bc85-13fb-4dd0-a8b1-fdf56ccac1a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:21:46.396664 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-69e26580-a97b-430d-94ec-7241c9face31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dfbf85b9e5b140ceba1fe0510ee1109f'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:46.404135 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1139/2260] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:21:46 2022] DELETE /identity/v3/users/dfbf85b9e5b140ceba1fe0510ee1109f => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:21:46.408843 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f34744-fbe3-4263-b04f-af8b7339378f None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:46.409293 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f34744-fbe3-4263-b04f-af8b7339378f None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:46.409711 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f34744-fbe3-4263-b04f-af8b7339378f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:46.439365 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-e0f34744-fbe3-4263-b04f-af8b7339378f None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:21:46.440844 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-e0f34744-fbe3-4263-b04f-af8b7339378f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:21:46.519386 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1122/2261] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:21:46 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:21:46.530681 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d140bc85-13fb-4dd0-a8b1-fdf56ccac1a6 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:46.532131 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-d140bc85-13fb-4dd0-a8b1-fdf56ccac1a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:21:46.535846 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1140/2262] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:21:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:21:46.537268 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:21:46.545821 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f34744-fbe3-4263-b04f-af8b7339378f None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:46.547517 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-e0f34744-fbe3-4263-b04f-af8b7339378f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:21:46.617768 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d140bc85-13fb-4dd0-a8b1-fdf56ccac1a6 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:46.663737 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f34744-fbe3-4263-b04f-af8b7339378f None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:46.668326 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bb74a3-2758-47f8-8d5e-f7770b392d14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:46.669014 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bb74a3-2758-47f8-8d5e-f7770b392d14 service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:46.669363 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bb74a3-2758-47f8-8d5e-f7770b392d14 service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:46.669698 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bb74a3-2758-47f8-8d5e-f7770b392d14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:46.682505 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bb74a3-2758-47f8-8d5e-f7770b392d14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:46.684029 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bb74a3-2758-47f8-8d5e-f7770b392d14 service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:46.695999 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1141/2263] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:21:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 25 03:21:46.696183 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0854b123-0254-47ed-bd03-a3f7251f8161 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60193ba9e1c14df3bc9eebcc6f28f2d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:46.696650 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0854b123-0254-47ed-bd03-a3f7251f8161 service cinder] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:46.696991 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0854b123-0254-47ed-bd03-a3f7251f8161 service cinder] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:46.697334 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0854b123-0254-47ed-bd03-a3f7251f8161 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:46.776059 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0854b123-0254-47ed-bd03-a3f7251f8161 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:46.777693 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0854b123-0254-47ed-bd03-a3f7251f8161 service cinder] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:46.790727 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1123/2264] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:21:46 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 25 03:21:46.927406 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bb74a3-2758-47f8-8d5e-f7770b392d14 service neutron] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:46.939338 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bb74a3-2758-47f8-8d5e-f7770b392d14 service neutron] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:46.951103 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d18ddd3-7e35-4574-a71c-7ad730ed49e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60193ba9e1c14df3bc9eebcc6f28f2d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:46.951586 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d18ddd3-7e35-4574-a71c-7ad730ed49e1 service cinder] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:46.951866 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d18ddd3-7e35-4574-a71c-7ad730ed49e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:46.952141 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d18ddd3-7e35-4574-a71c-7ad730ed49e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:46.973719 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-8d18ddd3-7e35-4574-a71c-7ad730ed49e1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:21:47.060653 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d18ddd3-7e35-4574-a71c-7ad730ed49e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:47.062210 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d18ddd3-7e35-4574-a71c-7ad730ed49e1 service cinder] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:47.073174 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1142/2265] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:21:46 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:21:47.201711 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0854b123-0254-47ed-bd03-a3f7251f8161 service cinder] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:47.203741 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-0854b123-0254-47ed-bd03-a3f7251f8161 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:21:47.220798 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0854b123-0254-47ed-bd03-a3f7251f8161 service cinder] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:47.226937 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079316e6-c0b7-4616-a510-81c493cf5e50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:47.227597 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079316e6-c0b7-4616-a510-81c493cf5e50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:47.227880 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079316e6-c0b7-4616-a510-81c493cf5e50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:47.228189 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079316e6-c0b7-4616-a510-81c493cf5e50 admin admin] PATH_INFO: `/v3/projects/91a9a2ba30f6433d976f82cba18a5373` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:47.229354 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079316e6-c0b7-4616-a510-81c493cf5e50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91a9a2ba30f6433d976f82cba18a5373)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:47.231342 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079316e6-c0b7-4616-a510-81c493cf5e50 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:47.350960 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-079316e6-c0b7-4616-a510-81c493cf5e50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91a9a2ba30f6433d976f82cba18a5373', 'request_id': 'req-079316e6-c0b7-4616-a510-81c493cf5e50'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:47.416260 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-079316e6-c0b7-4616-a510-81c493cf5e50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91a9a2ba30f6433d976f82cba18a5373', 'request_id': 'req-079316e6-c0b7-4616-a510-81c493cf5e50'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:47.423217 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-079316e6-c0b7-4616-a510-81c493cf5e50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91a9a2ba30f6433d976f82cba18a5373', 'request_id': 'req-079316e6-c0b7-4616-a510-81c493cf5e50'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:47.424886 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:21:47.442624 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-079316e6-c0b7-4616-a510-81c493cf5e50 admin admin] The token cache is being invalidate because project 91a9a2ba30f6433d976f82cba18a5373 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81995) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:21:47.442969 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-079316e6-c0b7-4616-a510-81c493cf5e50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:47.444216 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-079316e6-c0b7-4616-a510-81c493cf5e50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:47.449917 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1124/2266] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:21:47 2022] DELETE /identity/v3/projects/91a9a2ba30f6433d976f82cba18a5373 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:21:47.453542 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace409c4-ddbc-439b-a0f4-3dd1bd73a04d None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:47.454064 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace409c4-ddbc-439b-a0f4-3dd1bd73a04d None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:47.454191 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace409c4-ddbc-439b-a0f4-3dd1bd73a04d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:47.473531 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-ace409c4-ddbc-439b-a0f4-3dd1bd73a04d None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:21:47.474781 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-ace409c4-ddbc-439b-a0f4-3dd1bd73a04d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:21:47.494866 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:21:47.541886 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1143/2267] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:21:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:21:47.549394 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab81791-88fe-4bbf-8a13-13abbf84f6fe None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:47.549650 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab81791-88fe-4bbf-8a13-13abbf84f6fe None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:47.549934 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab81791-88fe-4bbf-8a13-13abbf84f6fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:47.570615 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-cab81791-88fe-4bbf-8a13-13abbf84f6fe None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:21:47.571831 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-cab81791-88fe-4bbf-8a13-13abbf84f6fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:21:47.588655 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1125/2268] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:21:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:21:47.590335 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace409c4-ddbc-439b-a0f4-3dd1bd73a04d None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:47.600801 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace409c4-ddbc-439b-a0f4-3dd1bd73a04d None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:47.606566 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6edfbd7-7484-4220-a3fb-e403a7c485df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:47.607933 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6edfbd7-7484-4220-a3fb-e403a7c485df admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:47.608421 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6edfbd7-7484-4220-a3fb-e403a7c485df admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:47.608836 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6edfbd7-7484-4220-a3fb-e403a7c485df admin admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:47.609694 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6edfbd7-7484-4220-a3fb-e403a7c485df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:21:47.610141 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6edfbd7-7484-4220-a3fb-e403a7c485df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:47.611927 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6edfbd7-7484-4220-a3fb-e403a7c485df admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:47.629408 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1144/2269] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Aug 25 03:21:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:21:47.630268 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab81791-88fe-4bbf-8a13-13abbf84f6fe None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:47.649431 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab81791-88fe-4bbf-8a13-13abbf84f6fe None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:47.654181 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1df1aae-a0e0-4015-86de-64e68d89813d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:47.654833 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1df1aae-a0e0-4015-86de-64e68d89813d admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:47.655242 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1df1aae-a0e0-4015-86de-64e68d89813d admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:47.655526 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1df1aae-a0e0-4015-86de-64e68d89813d admin admin] PATH_INFO: `/v3/projects` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:47.656164 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1df1aae-a0e0-4015-86de-64e68d89813d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:47.658205 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1df1aae-a0e0-4015-86de-64e68d89813d admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:47.759935 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1126/2270] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:21:47 2022] POST /identity/v3/projects => generated 371 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:21:47.779970 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6edfbd7-7484-4220-a3fb-e403a7c485df admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:47.795191 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6edfbd7-7484-4220-a3fb-e403a7c485df admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:47.859572 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a9e7d6-25a3-40ef-8a89-ad3bca489ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:47.860225 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a9e7d6-25a3-40ef-8a89-ad3bca489ddd admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:47.860502 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a9e7d6-25a3-40ef-8a89-ad3bca489ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:47.860749 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a9e7d6-25a3-40ef-8a89-ad3bca489ddd admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:47.861380 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a9e7d6-25a3-40ef-8a89-ad3bca489ddd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:47.863388 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a9e7d6-25a3-40ef-8a89-ad3bca489ddd admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:47.968776 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1145/2271] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:21:47 2022] POST /identity/v3/users => generated 286 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:21:47.973107 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1df1aae-a0e0-4015-86de-64e68d89813d admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:47.984318 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1df1aae-a0e0-4015-86de-64e68d89813d admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:47.989467 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70637e5b-fb43-4b51-99c3-9abf772ecb2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:47.990150 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70637e5b-fb43-4b51-99c3-9abf772ecb2c admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:47.990581 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70637e5b-fb43-4b51-99c3-9abf772ecb2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:47.990927 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70637e5b-fb43-4b51-99c3-9abf772ecb2c admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:47.991646 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70637e5b-fb43-4b51-99c3-9abf772ecb2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:47.993307 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70637e5b-fb43-4b51-99c3-9abf772ecb2c admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:48.003546 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1127/2272] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:21:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:21:48.009599 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a9e7d6-25a3-40ef-8a89-ad3bca489ddd admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:48.020278 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a9e7d6-25a3-40ef-8a89-ad3bca489ddd admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:48.025888 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedecc1c-9935-4d39-b2ef-7bf382dfe461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:48.026744 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedecc1c-9935-4d39-b2ef-7bf382dfe461 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:48.030612 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedecc1c-9935-4d39-b2ef-7bf382dfe461 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:48.031258 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedecc1c-9935-4d39-b2ef-7bf382dfe461 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:48.032289 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedecc1c-9935-4d39-b2ef-7bf382dfe461 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:48.034064 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedecc1c-9935-4d39-b2ef-7bf382dfe461 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:48.051657 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1146/2273] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:21:48 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:21:48.056153 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70637e5b-fb43-4b51-99c3-9abf772ecb2c admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:48.065963 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70637e5b-fb43-4b51-99c3-9abf772ecb2c admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:48.071141 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1883dd9-680b-41d6-a5b0-2e9f5d745048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:48.071791 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1883dd9-680b-41d6-a5b0-2e9f5d745048 admin admin] REQUEST_METHOD: `PUT` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:48.072071 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1883dd9-680b-41d6-a5b0-2e9f5d745048 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:48.072345 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1883dd9-680b-41d6-a5b0-2e9f5d745048 admin admin] PATH_INFO: `/v3/projects/8a5fd93958cf446282cae6099e5e293d/users/16652247622e4f40965fae5ee3c390a4/roles/3255f77192524a16bc405173106871fc` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:48.087776 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1883dd9-680b-41d6-a5b0-2e9f5d745048 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a5fd93958cf446282cae6099e5e293d, user_id=16652247622e4f40965fae5ee3c390a4, role_id=3255f77192524a16bc405173106871fc)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:48.090484 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1883dd9-680b-41d6-a5b0-2e9f5d745048 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:48.170964 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1128/2274] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:21:48 2022] PUT /identity/v3/projects/8a5fd93958cf446282cae6099e5e293d/users/16652247622e4f40965fae5ee3c390a4/roles/3255f77192524a16bc405173106871fc => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:21:48.181464 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e939298-0675-4c70-a929-7b0baafaf8f5 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:48.181751 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e939298-0675-4c70-a929-7b0baafaf8f5 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:48.182081 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e939298-0675-4c70-a929-7b0baafaf8f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:48.213308 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-0e939298-0675-4c70-a929-7b0baafaf8f5 None None] MFA Rules not processed for user `16652247622e4f40965fae5ee3c390a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:21:48.214603 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-0e939298-0675-4c70-a929-7b0baafaf8f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:21:48.283926 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1147/2275] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:21:48 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:21:48.296056 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1883dd9-680b-41d6-a5b0-2e9f5d745048 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:48.297789 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-e1883dd9-680b-41d6-a5b0-2e9f5d745048 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:21:48.356371 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1883dd9-680b-41d6-a5b0-2e9f5d745048 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:48.360016 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35095e9-f75a-4be7-9ca0-4dfb4fdfdafc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60193ba9e1c14df3bc9eebcc6f28f2d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:48.360625 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35095e9-f75a-4be7-9ca0-4dfb4fdfdafc service cinder] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:48.360906 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35095e9-f75a-4be7-9ca0-4dfb4fdfdafc service cinder] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:48.361180 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35095e9-f75a-4be7-9ca0-4dfb4fdfdafc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:48.370008 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35095e9-f75a-4be7-9ca0-4dfb4fdfdafc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:48.371490 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35095e9-f75a-4be7-9ca0-4dfb4fdfdafc service cinder] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:48.380082 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1129/2276] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:21:48 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 25 03:21:48.405459 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e939298-0675-4c70-a929-7b0baafaf8f5 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:48.424633 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:21:48.476235 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e939298-0675-4c70-a929-7b0baafaf8f5 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:48.495426 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3d4205-a58e-4d3b-8cf5-f9c0ff62ffc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:48.496345 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3d4205-a58e-4d3b-8cf5-f9c0ff62ffc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:48.496631 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3d4205-a58e-4d3b-8cf5-f9c0ff62ffc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:48.497009 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3d4205-a58e-4d3b-8cf5-f9c0ff62ffc7 admin admin] PATH_INFO: `/v3/users/16652247622e4f40965fae5ee3c390a4` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:48.498143 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3d4205-a58e-4d3b-8cf5-f9c0ff62ffc7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=16652247622e4f40965fae5ee3c390a4)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:48.500133 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3d4205-a58e-4d3b-8cf5-f9c0ff62ffc7 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:48.687838 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-4b3d4205-a58e-4d3b-8cf5-f9c0ff62ffc7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16652247622e4f40965fae5ee3c390a4'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:48.799108 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-4b3d4205-a58e-4d3b-8cf5-f9c0ff62ffc7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16652247622e4f40965fae5ee3c390a4'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:48.799731 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-4b3d4205-a58e-4d3b-8cf5-f9c0ff62ffc7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16652247622e4f40965fae5ee3c390a4'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:48.807749 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-4b3d4205-a58e-4d3b-8cf5-f9c0ff62ffc7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16652247622e4f40965fae5ee3c390a4'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:48.820238 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-4b3d4205-a58e-4d3b-8cf5-f9c0ff62ffc7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16652247622e4f40965fae5ee3c390a4'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:48.827582 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1148/2277] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:21:48 2022] DELETE /identity/v3/users/16652247622e4f40965fae5ee3c390a4 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:21:48.832144 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da76961-8f2b-4ba6-9c4e-075a3064c88e None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:48.832788 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da76961-8f2b-4ba6-9c4e-075a3064c88e None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:48.833153 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da76961-8f2b-4ba6-9c4e-075a3064c88e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:48.852839 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-4da76961-8f2b-4ba6-9c4e-075a3064c88e None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:21:48.857876 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-4da76961-8f2b-4ba6-9c4e-075a3064c88e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:21:48.922403 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1130/2278] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:21:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:21:48.933224 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3d4205-a58e-4d3b-8cf5-f9c0ff62ffc7 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:48.940643 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-4b3d4205-a58e-4d3b-8cf5-f9c0ff62ffc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:21:49.001077 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3d4205-a58e-4d3b-8cf5-f9c0ff62ffc7 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:49.004888 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a3b6ce-6db9-4370-8777-b4295a034993 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:49.005571 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a3b6ce-6db9-4370-8777-b4295a034993 service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:49.005924 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a3b6ce-6db9-4370-8777-b4295a034993 service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:49.006334 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a3b6ce-6db9-4370-8777-b4295a034993 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:49.015563 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a3b6ce-6db9-4370-8777-b4295a034993 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:49.019073 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a3b6ce-6db9-4370-8777-b4295a034993 service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:49.029089 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1149/2279] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:21:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:21:49.457712 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da76961-8f2b-4ba6-9c4e-075a3064c88e None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:49.459192 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-4da76961-8f2b-4ba6-9c4e-075a3064c88e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:21:49.472065 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da76961-8f2b-4ba6-9c4e-075a3064c88e None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:49.476733 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327442f8-2fdb-4607-b319-3789ffc7b5d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:49.477477 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327442f8-2fdb-4607-b319-3789ffc7b5d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:49.477766 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327442f8-2fdb-4607-b319-3789ffc7b5d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:49.478043 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327442f8-2fdb-4607-b319-3789ffc7b5d3 admin admin] PATH_INFO: `/v3/projects/8a5fd93958cf446282cae6099e5e293d` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:49.480022 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327442f8-2fdb-4607-b319-3789ffc7b5d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a5fd93958cf446282cae6099e5e293d)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:49.481999 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327442f8-2fdb-4607-b319-3789ffc7b5d3 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:49.606513 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:21:49.622587 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-327442f8-2fdb-4607-b319-3789ffc7b5d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a5fd93958cf446282cae6099e5e293d', 'request_id': 'req-327442f8-2fdb-4607-b319-3789ffc7b5d3'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:49.690227 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-327442f8-2fdb-4607-b319-3789ffc7b5d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a5fd93958cf446282cae6099e5e293d', 'request_id': 'req-327442f8-2fdb-4607-b319-3789ffc7b5d3'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:49.696594 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-327442f8-2fdb-4607-b319-3789ffc7b5d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a5fd93958cf446282cae6099e5e293d', 'request_id': 'req-327442f8-2fdb-4607-b319-3789ffc7b5d3'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:49.707763 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-327442f8-2fdb-4607-b319-3789ffc7b5d3 admin admin] The token cache is being invalidate because project 8a5fd93958cf446282cae6099e5e293d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81995) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:21:49.708060 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-327442f8-2fdb-4607-b319-3789ffc7b5d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:49.708681 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-327442f8-2fdb-4607-b319-3789ffc7b5d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:21:49.710841 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1131/2280] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:21:49 2022] DELETE /identity/v3/projects/8a5fd93958cf446282cae6099e5e293d => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:21:49.715876 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fa9fbe-c158-42ab-a5ef-ef8735f9934b None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:49.716324 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fa9fbe-c158-42ab-a5ef-ef8735f9934b None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:49.716416 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fa9fbe-c158-42ab-a5ef-ef8735f9934b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:49.735269 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-96fa9fbe-c158-42ab-a5ef-ef8735f9934b None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:21:49.736385 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-96fa9fbe-c158-42ab-a5ef-ef8735f9934b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:21:49.773961 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:21:49.798807 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1150/2281] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:21:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:21:49.808248 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b49a654-75a8-4b73-8e64-5914e6e1b223 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:49.808538 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b49a654-75a8-4b73-8e64-5914e6e1b223 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:49.808817 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b49a654-75a8-4b73-8e64-5914e6e1b223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:49.829920 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-3b49a654-75a8-4b73-8e64-5914e6e1b223 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:21:49.831861 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-3b49a654-75a8-4b73-8e64-5914e6e1b223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:21:49.839819 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1132/2282] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:21:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:21:49.844114 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fa9fbe-c158-42ab-a5ef-ef8735f9934b None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:49.854734 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fa9fbe-c158-42ab-a5ef-ef8735f9934b None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:49.859267 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306fc1f6-fb21-4bab-bbc7-3227a7e6b932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:49.859866 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306fc1f6-fb21-4bab-bbc7-3227a7e6b932 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:49.860140 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306fc1f6-fb21-4bab-bbc7-3227a7e6b932 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:49.860440 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306fc1f6-fb21-4bab-bbc7-3227a7e6b932 admin admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:49.861091 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306fc1f6-fb21-4bab-bbc7-3227a7e6b932 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:21:49.861388 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306fc1f6-fb21-4bab-bbc7-3227a7e6b932 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:49.863008 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306fc1f6-fb21-4bab-bbc7-3227a7e6b932 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:49.875352 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1151/2283] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Aug 25 03:21:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:21:49.879998 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b49a654-75a8-4b73-8e64-5914e6e1b223 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:49.889930 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b49a654-75a8-4b73-8e64-5914e6e1b223 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:49.894568 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eb688f-900e-435b-87b0-3674b335f11c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:49.895164 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eb688f-900e-435b-87b0-3674b335f11c admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:49.895444 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eb688f-900e-435b-87b0-3674b335f11c admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:49.895718 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eb688f-900e-435b-87b0-3674b335f11c admin admin] PATH_INFO: `/v3/projects` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:49.896337 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0eb688f-900e-435b-87b0-3674b335f11c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:49.898240 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0eb688f-900e-435b-87b0-3674b335f11c admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:50.046139 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1133/2284] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:21:49 2022] POST /identity/v3/projects => generated 379 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:21:50.046534 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306fc1f6-fb21-4bab-bbc7-3227a7e6b932 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:50.046534 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306fc1f6-fb21-4bab-bbc7-3227a7e6b932 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:50.090169 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cabeafb-e60a-4f73-84ae-e79108d1db52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:50.090817 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cabeafb-e60a-4f73-84ae-e79108d1db52 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:50.091100 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cabeafb-e60a-4f73-84ae-e79108d1db52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:50.091373 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cabeafb-e60a-4f73-84ae-e79108d1db52 admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:50.091993 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cabeafb-e60a-4f73-84ae-e79108d1db52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:50.093911 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cabeafb-e60a-4f73-84ae-e79108d1db52 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:50.158874 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1152/2285] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:21:50 2022] POST /identity/v3/users => generated 290 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:21:50.163487 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eb688f-900e-435b-87b0-3674b335f11c admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:50.173827 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eb688f-900e-435b-87b0-3674b335f11c admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:50.179070 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b994f02e-505c-4dde-8675-1511ab33973a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:50.179776 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b994f02e-505c-4dde-8675-1511ab33973a admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:50.180061 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b994f02e-505c-4dde-8675-1511ab33973a admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:50.180342 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b994f02e-505c-4dde-8675-1511ab33973a admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:50.181013 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b994f02e-505c-4dde-8675-1511ab33973a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:50.182647 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b994f02e-505c-4dde-8675-1511ab33973a admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:50.195960 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1134/2286] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:21:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:21:50.197139 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cabeafb-e60a-4f73-84ae-e79108d1db52 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:50.207522 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cabeafb-e60a-4f73-84ae-e79108d1db52 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:50.212121 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2094fa6-d44c-4ea5-8647-aad3791354f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:50.212720 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2094fa6-d44c-4ea5-8647-aad3791354f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:50.213001 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2094fa6-d44c-4ea5-8647-aad3791354f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:50.213605 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2094fa6-d44c-4ea5-8647-aad3791354f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:50.213910 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2094fa6-d44c-4ea5-8647-aad3791354f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:50.215606 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2094fa6-d44c-4ea5-8647-aad3791354f9 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:50.227369 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1153/2287] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:21:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:21:50.230722 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b994f02e-505c-4dde-8675-1511ab33973a admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:50.241320 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b994f02e-505c-4dde-8675-1511ab33973a admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:50.245980 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e1aaaa-8ea8-49f2-8b86-f38cdacb7cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:50.246738 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e1aaaa-8ea8-49f2-8b86-f38cdacb7cc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:50.246951 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e1aaaa-8ea8-49f2-8b86-f38cdacb7cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:50.247244 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e1aaaa-8ea8-49f2-8b86-f38cdacb7cc0 admin admin] PATH_INFO: `/v3/projects/5d445535a4974186a835ab117109dca1/users/d0a0e9c327564a3db8b89ea706f0fb36/roles/3255f77192524a16bc405173106871fc` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:50.261363 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e1aaaa-8ea8-49f2-8b86-f38cdacb7cc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d445535a4974186a835ab117109dca1, user_id=d0a0e9c327564a3db8b89ea706f0fb36, role_id=3255f77192524a16bc405173106871fc)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:50.264043 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e1aaaa-8ea8-49f2-8b86-f38cdacb7cc0 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:50.335225 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1135/2288] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:21:50 2022] PUT /identity/v3/projects/5d445535a4974186a835ab117109dca1/users/d0a0e9c327564a3db8b89ea706f0fb36/roles/3255f77192524a16bc405173106871fc => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:21:50.340155 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f49126-9bca-451d-858f-18486e3a090e None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:50.340441 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f49126-9bca-451d-858f-18486e3a090e None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:50.340714 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f49126-9bca-451d-858f-18486e3a090e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:50.359402 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-d0f49126-9bca-451d-858f-18486e3a090e None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:21:50.360590 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-d0f49126-9bca-451d-858f-18486e3a090e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:21:50.418234 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1154/2289] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:21:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:21:50.427646 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e1aaaa-8ea8-49f2-8b86-f38cdacb7cc0 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:50.430000 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-b4e1aaaa-8ea8-49f2-8b86-f38cdacb7cc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:21:50.484654 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e1aaaa-8ea8-49f2-8b86-f38cdacb7cc0 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:50.488336 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253fbcd6-ecc1-42f0-9a6b-5247bbd9c879 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:50.488924 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253fbcd6-ecc1-42f0-9a6b-5247bbd9c879 service neutron] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:50.489205 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253fbcd6-ecc1-42f0-9a6b-5247bbd9c879 service neutron] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:50.489477 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253fbcd6-ecc1-42f0-9a6b-5247bbd9c879 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:50.498931 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253fbcd6-ecc1-42f0-9a6b-5247bbd9c879 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:50.500299 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253fbcd6-ecc1-42f0-9a6b-5247bbd9c879 service neutron] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:50.509534 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1136/2290] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:21:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:21:53.594619 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca71c3fb-4928-439f-9e84-30422a8efcee None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:53.597406 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca71c3fb-4928-439f-9e84-30422a8efcee None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:53.597406 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca71c3fb-4928-439f-9e84-30422a8efcee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:53.616225 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-ca71c3fb-4928-439f-9e84-30422a8efcee None None] MFA Rules not processed for user `712c35b765b64919aa3eaaf02892590d`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:21:53.617563 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-ca71c3fb-4928-439f-9e84-30422a8efcee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:21:53.672143 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:21:53.695629 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1155/2291] 10.222.0.58 () {58 vars in 1009 bytes} [Thu Aug 25 03:21:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:21:53.758228 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253fbcd6-ecc1-42f0-9a6b-5247bbd9c879 service neutron] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:53.762576 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-253fbcd6-ecc1-42f0-9a6b-5247bbd9c879 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:21:53.844052 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:21:53.865548 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253fbcd6-ecc1-42f0-9a6b-5247bbd9c879 service neutron] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:53.871831 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7f1dec-b29c-4bd7-9a97-ac620ae82a90 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5047e1a7cd2d416d90b2d8c5a6596522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:53.873086 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7f1dec-b29c-4bd7-9a97-ac620ae82a90 service swift] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:53.873612 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7f1dec-b29c-4bd7-9a97-ac620ae82a90 service swift] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:53.873976 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7f1dec-b29c-4bd7-9a97-ac620ae82a90 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:53.886454 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7f1dec-b29c-4bd7-9a97-ac620ae82a90 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:53.888437 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7f1dec-b29c-4bd7-9a97-ac620ae82a90 service swift] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:53.898781 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1137/2292] 10.222.0.58 () {60 vars in 1384 bytes} [Thu Aug 25 03:21:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 25 03:21:55.999044 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f686600-f1fe-4aab-89a7-caad6076ed6d None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:55.999534 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f686600-f1fe-4aab-89a7-caad6076ed6d None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:55.999633 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f686600-f1fe-4aab-89a7-caad6076ed6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:56.028746 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-4f686600-f1fe-4aab-89a7-caad6076ed6d None None] MFA Rules not processed for user `d0a0e9c327564a3db8b89ea706f0fb36`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:21:56.029849 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-4f686600-f1fe-4aab-89a7-caad6076ed6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:21:56.099544 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1156/2293] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:21:55 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:21:56.103942 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7f1dec-b29c-4bd7-9a97-ac620ae82a90 service swift] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:56.114953 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7f1dec-b29c-4bd7-9a97-ac620ae82a90 service swift] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:56.120244 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57c0154-0c3d-4715-a76f-096c875497ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:56.120857 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57c0154-0c3d-4715-a76f-096c875497ef admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:56.121142 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57c0154-0c3d-4715-a76f-096c875497ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:56.121418 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57c0154-0c3d-4715-a76f-096c875497ef admin admin] PATH_INFO: `/v3/projects` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:56.122051 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57c0154-0c3d-4715-a76f-096c875497ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:56.124324 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57c0154-0c3d-4715-a76f-096c875497ef admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:56.216465 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1138/2294] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:21:56 2022] POST /identity/v3/projects => generated 379 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:21:56.220272 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f686600-f1fe-4aab-89a7-caad6076ed6d None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:56.232574 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f686600-f1fe-4aab-89a7-caad6076ed6d None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:56.293758 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c53d06a-86cf-433b-94e7-6043a66e62f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:56.294423 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c53d06a-86cf-433b-94e7-6043a66e62f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:56.294707 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c53d06a-86cf-433b-94e7-6043a66e62f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:56.294986 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c53d06a-86cf-433b-94e7-6043a66e62f9 admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:56.295632 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c53d06a-86cf-433b-94e7-6043a66e62f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:56.297572 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c53d06a-86cf-433b-94e7-6043a66e62f9 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:56.382695 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1157/2295] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:21:56 2022] POST /identity/v3/users => generated 296 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:21:56.386698 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57c0154-0c3d-4715-a76f-096c875497ef admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:56.397121 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57c0154-0c3d-4715-a76f-096c875497ef admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:56.401872 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7139692-e633-42e5-aaa1-963fdad8f956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:56.402526 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7139692-e633-42e5-aaa1-963fdad8f956 admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:56.402847 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7139692-e633-42e5-aaa1-963fdad8f956 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:56.403126 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7139692-e633-42e5-aaa1-963fdad8f956 admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:56.403777 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7139692-e633-42e5-aaa1-963fdad8f956 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:56.405416 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7139692-e633-42e5-aaa1-963fdad8f956 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:56.422455 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1139/2296] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:21:56 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:21:56.426523 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c53d06a-86cf-433b-94e7-6043a66e62f9 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:56.437418 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c53d06a-86cf-433b-94e7-6043a66e62f9 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:56.442267 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6d6114-5ba8-4c4f-9584-ac9d188774f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:56.442958 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6d6114-5ba8-4c4f-9584-ac9d188774f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:56.443242 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6d6114-5ba8-4c4f-9584-ac9d188774f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:56.443520 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6d6114-5ba8-4c4f-9584-ac9d188774f1 admin admin] PATH_INFO: `/v3/projects/3cc897b906b243e395495bfed506497c/users/97897f84a40f4401929fe2746945b6e3/roles/62f19ea272b447ccaab1576cd9bf4fd9` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:56.459599 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6d6114-5ba8-4c4f-9584-ac9d188774f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3cc897b906b243e395495bfed506497c, user_id=97897f84a40f4401929fe2746945b6e3, role_id=62f19ea272b447ccaab1576cd9bf4fd9)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:56.462372 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6d6114-5ba8-4c4f-9584-ac9d188774f1 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:56.536467 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1158/2297] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:21:56 2022] PUT /identity/v3/projects/3cc897b906b243e395495bfed506497c/users/97897f84a40f4401929fe2746945b6e3/roles/62f19ea272b447ccaab1576cd9bf4fd9 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:21:56.540962 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7139692-e633-42e5-aaa1-963fdad8f956 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:56.551198 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7139692-e633-42e5-aaa1-963fdad8f956 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:56.608485 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e7c9bd-a158-40a6-ad0b-9e4d0a246ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:56.609091 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e7c9bd-a158-40a6-ad0b-9e4d0a246ce3 admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:56.609371 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e7c9bd-a158-40a6-ad0b-9e4d0a246ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:56.609648 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e7c9bd-a158-40a6-ad0b-9e4d0a246ce3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:56.610553 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e7c9bd-a158-40a6-ad0b-9e4d0a246ce3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:56.612177 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e7c9bd-a158-40a6-ad0b-9e4d0a246ce3 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:56.621810 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1140/2298] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:21:56 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:21:56.626117 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6d6114-5ba8-4c4f-9584-ac9d188774f1 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:56.637211 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6d6114-5ba8-4c4f-9584-ac9d188774f1 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:56.641742 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63358cf8-c33d-4db8-8792-18c50c783481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:56.642517 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63358cf8-c33d-4db8-8792-18c50c783481 admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:56.642801 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63358cf8-c33d-4db8-8792-18c50c783481 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:56.643077 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63358cf8-c33d-4db8-8792-18c50c783481 admin admin] PATH_INFO: `/v3/projects/3cc897b906b243e395495bfed506497c/users/97897f84a40f4401929fe2746945b6e3/roles/62f19ea272b447ccaab1576cd9bf4fd9` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:56.645142 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63358cf8-c33d-4db8-8792-18c50c783481 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3cc897b906b243e395495bfed506497c, user_id=97897f84a40f4401929fe2746945b6e3, role_id=62f19ea272b447ccaab1576cd9bf4fd9)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:56.648301 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63358cf8-c33d-4db8-8792-18c50c783481 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:56.659378 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1159/2299] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:21:56 2022] PUT /identity/v3/projects/3cc897b906b243e395495bfed506497c/users/97897f84a40f4401929fe2746945b6e3/roles/62f19ea272b447ccaab1576cd9bf4fd9 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:22:00.038045 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e7c9bd-a158-40a6-ad0b-9e4d0a246ce3 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:22:00.039998 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-08e7c9bd-a158-40a6-ad0b-9e4d0a246ce3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:22:00.068648 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e7c9bd-a158-40a6-ad0b-9e4d0a246ce3 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:22:00.284067 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782410fc-e405-44ce-b409-12dd628df61d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:22:00.284738 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782410fc-e405-44ce-b409-12dd628df61d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:22:00.285019 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782410fc-e405-44ce-b409-12dd628df61d admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:22:00.285295 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782410fc-e405-44ce-b409-12dd628df61d admin admin] PATH_INFO: `/v3/users/8490068a3789495783b1045e1ab12342` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:22:00.293029 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782410fc-e405-44ce-b409-12dd628df61d admin admin] RBAC: Authorizing `identity:delete_user(user_id=8490068a3789495783b1045e1ab12342)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:22:00.303780 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782410fc-e405-44ce-b409-12dd628df61d admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:22:00.635419 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-782410fc-e405-44ce-b409-12dd628df61d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8490068a3789495783b1045e1ab12342'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:22:00.639006 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:22:00.747255 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-782410fc-e405-44ce-b409-12dd628df61d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8490068a3789495783b1045e1ab12342'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:22:00.748356 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-782410fc-e405-44ce-b409-12dd628df61d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8490068a3789495783b1045e1ab12342'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:22:00.749162 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-782410fc-e405-44ce-b409-12dd628df61d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8490068a3789495783b1045e1ab12342'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:22:00.772525 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-782410fc-e405-44ce-b409-12dd628df61d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8490068a3789495783b1045e1ab12342'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:22:00.808675 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1141/2300] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:22:00 2022] DELETE /identity/v3/users/8490068a3789495783b1045e1ab12342 => generated 0 bytes in 771 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:22:00.813099 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36474556-11fc-4707-815f-a79ca4a6e08b None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:22:00.813798 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36474556-11fc-4707-815f-a79ca4a6e08b None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:22:00.813798 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36474556-11fc-4707-815f-a79ca4a6e08b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:22:00.844647 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-36474556-11fc-4707-815f-a79ca4a6e08b None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:22:00.845943 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-36474556-11fc-4707-815f-a79ca4a6e08b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:22:00.894520 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:22:01.009936 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1160/2301] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:22:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:22:01.045339 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782410fc-e405-44ce-b409-12dd628df61d admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:22:01.048089 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-782410fc-e405-44ce-b409-12dd628df61d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:22:01.194691 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782410fc-e405-44ce-b409-12dd628df61d admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:22:01.203929 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d955763-3428-4ee2-9264-dc4e1dffae27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:22:01.205356 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d955763-3428-4ee2-9264-dc4e1dffae27 service neutron] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:22:01.206529 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d955763-3428-4ee2-9264-dc4e1dffae27 service neutron] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:22:01.214813 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d955763-3428-4ee2-9264-dc4e1dffae27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:22:01.236555 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d955763-3428-4ee2-9264-dc4e1dffae27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:22:01.238127 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d955763-3428-4ee2-9264-dc4e1dffae27 service neutron] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:22:01.267923 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1142/2302] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:22:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:22:02.087966 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36474556-11fc-4707-815f-a79ca4a6e08b None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:22:02.091698 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-36474556-11fc-4707-815f-a79ca4a6e08b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:22:02.106750 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36474556-11fc-4707-815f-a79ca4a6e08b None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:22:02.113092 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb82d5de-369e-42f7-917b-1da24e8592db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:22:02.113972 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb82d5de-369e-42f7-917b-1da24e8592db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:22:02.114450 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb82d5de-369e-42f7-917b-1da24e8592db admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:22:02.120318 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb82d5de-369e-42f7-917b-1da24e8592db admin admin] PATH_INFO: `/v3/projects/4ae2b7c2698146cc83a6378704734a13` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:22:02.120318 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb82d5de-369e-42f7-917b-1da24e8592db admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ae2b7c2698146cc83a6378704734a13)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:22:02.120318 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb82d5de-369e-42f7-917b-1da24e8592db admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:22:02.296572 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-fb82d5de-369e-42f7-917b-1da24e8592db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ae2b7c2698146cc83a6378704734a13', 'request_id': 'req-fb82d5de-369e-42f7-917b-1da24e8592db'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:22:02.381783 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-fb82d5de-369e-42f7-917b-1da24e8592db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ae2b7c2698146cc83a6378704734a13', 'request_id': 'req-fb82d5de-369e-42f7-917b-1da24e8592db'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:22:02.389515 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-fb82d5de-369e-42f7-917b-1da24e8592db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ae2b7c2698146cc83a6378704734a13', 'request_id': 'req-fb82d5de-369e-42f7-917b-1da24e8592db'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:22:02.423772 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-fb82d5de-369e-42f7-917b-1da24e8592db admin admin] The token cache is being invalidate because project 4ae2b7c2698146cc83a6378704734a13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:22:02.424772 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-fb82d5de-369e-42f7-917b-1da24e8592db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:22:02.425076 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-fb82d5de-369e-42f7-917b-1da24e8592db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:22:02.428978 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1161/2303] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:22:02 2022] DELETE /identity/v3/projects/4ae2b7c2698146cc83a6378704734a13 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:22:02.438390 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d955763-3428-4ee2-9264-dc4e1dffae27 service neutron] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:22:02.485926 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d955763-3428-4ee2-9264-dc4e1dffae27 service neutron] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:22:02.607006 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cfb1ff-cd55-44fa-a1ba-1ac275d6d1b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:22:02.607685 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cfb1ff-cd55-44fa-a1ba-1ac275d6d1b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:22:02.607966 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cfb1ff-cd55-44fa-a1ba-1ac275d6d1b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:22:02.608315 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cfb1ff-cd55-44fa-a1ba-1ac275d6d1b2 admin admin] PATH_INFO: `/v3/users/dc986da9b803458fa1f77504c3822aeb` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:22:02.609832 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cfb1ff-cd55-44fa-a1ba-1ac275d6d1b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc986da9b803458fa1f77504c3822aeb)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:22:02.611894 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cfb1ff-cd55-44fa-a1ba-1ac275d6d1b2 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:22:02.817723 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-50cfb1ff-cd55-44fa-a1ba-1ac275d6d1b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc986da9b803458fa1f77504c3822aeb'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:22:02.902101 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-50cfb1ff-cd55-44fa-a1ba-1ac275d6d1b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc986da9b803458fa1f77504c3822aeb'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:22:02.905736 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-50cfb1ff-cd55-44fa-a1ba-1ac275d6d1b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc986da9b803458fa1f77504c3822aeb'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:22:02.908301 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-50cfb1ff-cd55-44fa-a1ba-1ac275d6d1b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc986da9b803458fa1f77504c3822aeb'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:22:02.938667 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:22:02.941141 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-50cfb1ff-cd55-44fa-a1ba-1ac275d6d1b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc986da9b803458fa1f77504c3822aeb'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:22:02.960054 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1143/2304] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:22:02 2022] DELETE /identity/v3/users/dc986da9b803458fa1f77504c3822aeb => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:22:03.530092 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158a0207-eac1-48e3-a224-9619ae46be96 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:22:03.531028 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158a0207-eac1-48e3-a224-9619ae46be96 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:22:03.531462 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158a0207-eac1-48e3-a224-9619ae46be96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:22:03.555117 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-158a0207-eac1-48e3-a224-9619ae46be96 None None] MFA Rules not processed for user `c7dc5019b26a4a8abf6436be3f1e56af`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:22:03.557106 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-158a0207-eac1-48e3-a224-9619ae46be96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:22:03.718678 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1162/2305] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:22:03 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:22:03.976130 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cfb1ff-cd55-44fa-a1ba-1ac275d6d1b2 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:22:03.976130 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-50cfb1ff-cd55-44fa-a1ba-1ac275d6d1b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:22:04.081791 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cfb1ff-cd55-44fa-a1ba-1ac275d6d1b2 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:22:04.089454 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc50f57-3cc3-4585-90f3-049889eb7e3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:22:04.090301 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc50f57-3cc3-4585-90f3-049889eb7e3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:22:04.090631 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc50f57-3cc3-4585-90f3-049889eb7e3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:22:04.090959 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc50f57-3cc3-4585-90f3-049889eb7e3c admin admin] PATH_INFO: `/v3/projects/c48b517b40e945a1aea2a25d53be49ce` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:22:04.092237 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc50f57-3cc3-4585-90f3-049889eb7e3c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c48b517b40e945a1aea2a25d53be49ce)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:22:04.094667 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc50f57-3cc3-4585-90f3-049889eb7e3c admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:22:04.233775 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-3cc50f57-3cc3-4585-90f3-049889eb7e3c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c48b517b40e945a1aea2a25d53be49ce', 'request_id': 'req-3cc50f57-3cc3-4585-90f3-049889eb7e3c'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:22:04.295712 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-3cc50f57-3cc3-4585-90f3-049889eb7e3c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c48b517b40e945a1aea2a25d53be49ce', 'request_id': 'req-3cc50f57-3cc3-4585-90f3-049889eb7e3c'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:22:04.301868 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158a0207-eac1-48e3-a224-9619ae46be96 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:22:04.302351 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-158a0207-eac1-48e3-a224-9619ae46be96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:22:04.304133 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-3cc50f57-3cc3-4585-90f3-049889eb7e3c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c48b517b40e945a1aea2a25d53be49ce', 'request_id': 'req-3cc50f57-3cc3-4585-90f3-049889eb7e3c'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:22:04.324078 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158a0207-eac1-48e3-a224-9619ae46be96 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:22:04.328282 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-3cc50f57-3cc3-4585-90f3-049889eb7e3c admin admin] The token cache is being invalidate because project c48b517b40e945a1aea2a25d53be49ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81995) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:22:04.328583 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-3cc50f57-3cc3-4585-90f3-049889eb7e3c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:22:04.330529 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-3cc50f57-3cc3-4585-90f3-049889eb7e3c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:22:04.343778 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1144/2306] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:22:03 2022] DELETE /identity/v3/projects/c48b517b40e945a1aea2a25d53be49ce => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:22:04.350909 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d040f7-0933-4b18-a3f1-3b5b62470766 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:22:04.351193 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d040f7-0933-4b18-a3f1-3b5b62470766 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:22:04.351620 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d040f7-0933-4b18-a3f1-3b5b62470766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:22:04.403275 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-28d040f7-0933-4b18-a3f1-3b5b62470766 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:22:04.403605 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-28d040f7-0933-4b18-a3f1-3b5b62470766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:22:04.452505 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeed20b-1176-4e5d-8020-d302f7e10c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:22:04.453727 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeed20b-1176-4e5d-8020-d302f7e10c58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:22:04.454268 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeed20b-1176-4e5d-8020-d302f7e10c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:22:04.455681 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeed20b-1176-4e5d-8020-d302f7e10c58 admin admin] PATH_INFO: `/v3/users/2c484664e87245b5903d15e67e08ae15` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:22:04.460012 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eeed20b-1176-4e5d-8020-d302f7e10c58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c484664e87245b5903d15e67e08ae15)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:22:04.465905 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eeed20b-1176-4e5d-8020-d302f7e10c58 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:22:04.473950 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:22:04.548707 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1145/2307] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:22:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:22:04.559976 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4d37ff-f607-40a2-8cc0-d6a465238a45 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:22:04.560812 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4d37ff-f607-40a2-8cc0-d6a465238a45 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:22:04.561205 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4d37ff-f607-40a2-8cc0-d6a465238a45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:22:04.596747 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-7f4d37ff-f607-40a2-8cc0-d6a465238a45 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:22:04.602384 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-7f4d37ff-f607-40a2-8cc0-d6a465238a45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:22:04.624619 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1146/2308] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:22:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:22:04.625761 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4d37ff-f607-40a2-8cc0-d6a465238a45 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:22:04.639477 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4d37ff-f607-40a2-8cc0-d6a465238a45 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:22:04.645636 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a240ad1-3b8e-43b6-a92a-f0107c27e460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:22:04.646626 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a240ad1-3b8e-43b6-a92a-f0107c27e460 admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:22:04.646975 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a240ad1-3b8e-43b6-a92a-f0107c27e460 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:22:04.647318 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a240ad1-3b8e-43b6-a92a-f0107c27e460 admin admin] PATH_INFO: `/v3/domains` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:22:04.648054 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a240ad1-3b8e-43b6-a92a-f0107c27e460 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:22:04.648428 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a240ad1-3b8e-43b6-a92a-f0107c27e460 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:22:04.653975 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a240ad1-3b8e-43b6-a92a-f0107c27e460 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:22:04.666361 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:22:04.678106 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1147/2309] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Aug 25 03:22:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:22:04.679269 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a240ad1-3b8e-43b6-a92a-f0107c27e460 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:22:04.702439 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a240ad1-3b8e-43b6-a92a-f0107c27e460 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:22:04.718648 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-8eeed20b-1176-4e5d-8020-d302f7e10c58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c484664e87245b5903d15e67e08ae15'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:22:04.723479 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419783ec-849b-46b3-ada3-9b41626d7734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:22:04.725009 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419783ec-849b-46b3-ada3-9b41626d7734 admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:22:04.726810 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419783ec-849b-46b3-ada3-9b41626d7734 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:22:04.728306 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419783ec-849b-46b3-ada3-9b41626d7734 admin admin] PATH_INFO: `/v3/projects` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:22:04.729619 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419783ec-849b-46b3-ada3-9b41626d7734 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:22:04.732321 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419783ec-849b-46b3-ada3-9b41626d7734 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:22:04.798671 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-8eeed20b-1176-4e5d-8020-d302f7e10c58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c484664e87245b5903d15e67e08ae15'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:22:04.798671 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-8eeed20b-1176-4e5d-8020-d302f7e10c58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c484664e87245b5903d15e67e08ae15'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:22:04.801844 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-8eeed20b-1176-4e5d-8020-d302f7e10c58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c484664e87245b5903d15e67e08ae15'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:22:04.822495 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-8eeed20b-1176-4e5d-8020-d302f7e10c58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c484664e87245b5903d15e67e08ae15'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:22:04.833050 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1163/2310] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:22:04 2022] DELETE /identity/v3/users/2c484664e87245b5903d15e67e08ae15 => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:22:04.838069 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226cd7fe-ba9b-47f9-96df-a5a13a44fbc5 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:22:04.838596 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226cd7fe-ba9b-47f9-96df-a5a13a44fbc5 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:22:04.840161 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226cd7fe-ba9b-47f9-96df-a5a13a44fbc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:22:04.879924 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1148/2311] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:22:04 2022] POST /identity/v3/projects => generated 375 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:22:04.888901 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419783ec-849b-46b3-ada3-9b41626d7734 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:22:04.889606 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-419783ec-849b-46b3-ada3-9b41626d7734 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:22:04.892081 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-226cd7fe-ba9b-47f9-96df-a5a13a44fbc5 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:22:04.893531 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-226cd7fe-ba9b-47f9-96df-a5a13a44fbc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:22:04.994198 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1164/2312] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:22:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:22:05.022401 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226cd7fe-ba9b-47f9-96df-a5a13a44fbc5 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:22:05.024721 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-226cd7fe-ba9b-47f9-96df-a5a13a44fbc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:22:05.077681 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419783ec-849b-46b3-ada3-9b41626d7734 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:22:05.085980 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafb3dba-37c5-4c18-b786-8c9578f56bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:22:05.086988 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafb3dba-37c5-4c18-b786-8c9578f56bbd admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:22:05.087786 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafb3dba-37c5-4c18-b786-8c9578f56bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:22:05.088118 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafb3dba-37c5-4c18-b786-8c9578f56bbd admin admin] PATH_INFO: `/v3/users` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:22:05.088860 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafb3dba-37c5-4c18-b786-8c9578f56bbd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:22:05.091798 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafb3dba-37c5-4c18-b786-8c9578f56bbd admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:22:05.170293 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226cd7fe-ba9b-47f9-96df-a5a13a44fbc5 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:22:05.178676 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d9b10c-ea1b-4d07-8886-0390ade24a74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:22:05.179413 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d9b10c-ea1b-4d07-8886-0390ade24a74 service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:22:05.203557 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1149/2313] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:22:04 2022] POST /identity/v3/users => generated 288 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:22:05.203759 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d9b10c-ea1b-4d07-8886-0390ade24a74 service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:22:05.203759 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d9b10c-ea1b-4d07-8886-0390ade24a74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:22:05.205050 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafb3dba-37c5-4c18-b786-8c9578f56bbd admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:22:05.233937 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d9b10c-ea1b-4d07-8886-0390ade24a74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:22:05.234421 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafb3dba-37c5-4c18-b786-8c9578f56bbd admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:22:05.236480 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d9b10c-ea1b-4d07-8886-0390ade24a74 service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:22:05.271910 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42ac67d-623e-4385-a617-f922d90f564f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:22:05.272708 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42ac67d-623e-4385-a617-f922d90f564f admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:22:05.272939 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42ac67d-623e-4385-a617-f922d90f564f admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:22:05.273220 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42ac67d-623e-4385-a617-f922d90f564f admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:22:05.273922 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42ac67d-623e-4385-a617-f922d90f564f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:22:05.276794 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42ac67d-623e-4385-a617-f922d90f564f admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:22:05.290052 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1165/2314] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:22:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:22:05.316744 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1150/2315] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:22:05 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:22:05.322321 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d9b10c-ea1b-4d07-8886-0390ade24a74 service neutron] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:22:05.344131 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d9b10c-ea1b-4d07-8886-0390ade24a74 service neutron] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:22:05.352172 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bab051-7b17-4944-9796-f91d8fa17b5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:22:05.352172 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bab051-7b17-4944-9796-f91d8fa17b5e admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:22:05.352172 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bab051-7b17-4944-9796-f91d8fa17b5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:22:05.352958 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bab051-7b17-4944-9796-f91d8fa17b5e admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:22:05.353526 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bab051-7b17-4944-9796-f91d8fa17b5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:22:05.355870 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bab051-7b17-4944-9796-f91d8fa17b5e admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:22:05.370387 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1166/2316] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:22:05 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:22:05.374326 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42ac67d-623e-4385-a617-f922d90f564f admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:22:05.387818 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42ac67d-623e-4385-a617-f922d90f564f admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:22:05.397655 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ecead6-bcbb-4d84-8463-0a9334581ab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:22:05.398057 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ecead6-bcbb-4d84-8463-0a9334581ab7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:22:05.398665 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ecead6-bcbb-4d84-8463-0a9334581ab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:22:05.399107 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ecead6-bcbb-4d84-8463-0a9334581ab7 admin admin] PATH_INFO: `/v3/projects/b133e68717b94a428ea3d78e3aaee0fb/users/fe34364c80b24b3698593cb2ca9ed1e5/roles/3255f77192524a16bc405173106871fc` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:22:05.426930 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ecead6-bcbb-4d84-8463-0a9334581ab7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b133e68717b94a428ea3d78e3aaee0fb, user_id=fe34364c80b24b3698593cb2ca9ed1e5, role_id=3255f77192524a16bc405173106871fc)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:22:05.433467 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ecead6-bcbb-4d84-8463-0a9334581ab7 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:22:05.537382 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1151/2317] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:22:05 2022] PUT /identity/v3/projects/b133e68717b94a428ea3d78e3aaee0fb/users/fe34364c80b24b3698593cb2ca9ed1e5/roles/3255f77192524a16bc405173106871fc => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:22:05.548223 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f80d346-57ef-4b84-bc09-31c404139f4b None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:22:05.548538 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f80d346-57ef-4b84-bc09-31c404139f4b None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:22:05.548796 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f80d346-57ef-4b84-bc09-31c404139f4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:22:05.620161 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:22:05.626843 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-8f80d346-57ef-4b84-bc09-31c404139f4b None None] MFA Rules not processed for user `fe34364c80b24b3698593cb2ca9ed1e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:22:05.628795 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-8f80d346-57ef-4b84-bc09-31c404139f4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:22:05.805652 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1167/2318] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:22:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:22:05.810471 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ecead6-bcbb-4d84-8463-0a9334581ab7 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:22:06.010186 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ecead6-bcbb-4d84-8463-0a9334581ab7 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:22:06.015643 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4284fbcc-811a-4604-a7cd-0fc35859ac30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:22:06.016082 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4284fbcc-811a-4604-a7cd-0fc35859ac30 admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:22:06.016307 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4284fbcc-811a-4604-a7cd-0fc35859ac30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:22:06.017127 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4284fbcc-811a-4604-a7cd-0fc35859ac30 admin admin] PATH_INFO: `/v3/projects` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:22:06.017867 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4284fbcc-811a-4604-a7cd-0fc35859ac30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:22:06.020161 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4284fbcc-811a-4604-a7cd-0fc35859ac30 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:22:06.239307 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f80d346-57ef-4b84-bc09-31c404139f4b None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:22:06.242438 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-8f80d346-57ef-4b84-bc09-31c404139f4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:22:06.252581 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1152/2319] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:22:05 2022] POST /identity/v3/projects => generated 375 bytes in 442 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:22:06.256064 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4284fbcc-811a-4604-a7cd-0fc35859ac30 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:22:06.309605 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f80d346-57ef-4b84-bc09-31c404139f4b None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:22:06.315160 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:22:06.421449 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4284fbcc-811a-4604-a7cd-0fc35859ac30 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:22:06.436348 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3914260-6df9-4863-b236-cacabf5cce23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:22:06.436348 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3914260-6df9-4863-b236-cacabf5cce23 admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:22:06.436348 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3914260-6df9-4863-b236-cacabf5cce23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:22:06.436348 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3914260-6df9-4863-b236-cacabf5cce23 admin admin] PATH_INFO: `/v3/users` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:22:06.436781 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3914260-6df9-4863-b236-cacabf5cce23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:22:06.439971 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3914260-6df9-4863-b236-cacabf5cce23 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:22:06.448954 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ed2e63-3b6e-48c3-a84b-b357d3e18a7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:22:06.449750 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ed2e63-3b6e-48c3-a84b-b357d3e18a7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:22:06.450141 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ed2e63-3b6e-48c3-a84b-b357d3e18a7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:22:06.455475 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ed2e63-3b6e-48c3-a84b-b357d3e18a7a admin admin] PATH_INFO: `/v3/projects/7e671c3bce4a4875b48ec740d026d5af` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:22:06.456914 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ed2e63-3b6e-48c3-a84b-b357d3e18a7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e671c3bce4a4875b48ec740d026d5af)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:22:06.459924 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ed2e63-3b6e-48c3-a84b-b357d3e18a7a admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:22:06.548629 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1153/2320] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:22:06 2022] POST /identity/v3/users => generated 294 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:22:06.552933 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3914260-6df9-4863-b236-cacabf5cce23 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:22:06.573850 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3914260-6df9-4863-b236-cacabf5cce23 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:22:06.624455 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d50bfd-930b-4dc6-95c5-80d115d6ea97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:22:06.628839 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d50bfd-930b-4dc6-95c5-80d115d6ea97 admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:22:06.637342 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d50bfd-930b-4dc6-95c5-80d115d6ea97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:22:06.639517 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d50bfd-930b-4dc6-95c5-80d115d6ea97 admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:22:06.644187 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d50bfd-930b-4dc6-95c5-80d115d6ea97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:22:06.648415 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d50bfd-930b-4dc6-95c5-80d115d6ea97 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:22:06.651664 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-d8ed2e63-3b6e-48c3-a84b-b357d3e18a7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e671c3bce4a4875b48ec740d026d5af', 'request_id': 'req-d8ed2e63-3b6e-48c3-a84b-b357d3e18a7a'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:22:06.681499 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1154/2321] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:22:06 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 25 03:22:06.683156 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d50bfd-930b-4dc6-95c5-80d115d6ea97 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:22:06.685551 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-c7d50bfd-930b-4dc6-95c5-80d115d6ea97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:22:06.758016 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-d8ed2e63-3b6e-48c3-a84b-b357d3e18a7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e671c3bce4a4875b48ec740d026d5af', 'request_id': 'req-d8ed2e63-3b6e-48c3-a84b-b357d3e18a7a'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:22:06.764872 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-d8ed2e63-3b6e-48c3-a84b-b357d3e18a7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e671c3bce4a4875b48ec740d026d5af', 'request_id': 'req-d8ed2e63-3b6e-48c3-a84b-b357d3e18a7a'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:22:06.779161 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-d8ed2e63-3b6e-48c3-a84b-b357d3e18a7a admin admin] The token cache is being invalidate because project 7e671c3bce4a4875b48ec740d026d5af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:22:06.779766 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-d8ed2e63-3b6e-48c3-a84b-b357d3e18a7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:22:06.780794 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-d8ed2e63-3b6e-48c3-a84b-b357d3e18a7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:22:06.782666 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1168/2322] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:22:06 2022] DELETE /identity/v3/projects/7e671c3bce4a4875b48ec740d026d5af => generated 0 bytes in 544 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:22:06.783581 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ed2e63-3b6e-48c3-a84b-b357d3e18a7a admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:22:06.785272 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-d8ed2e63-3b6e-48c3-a84b-b357d3e18a7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:22:06.806783 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d50bfd-930b-4dc6-95c5-80d115d6ea97 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:22:06.855084 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:22:06.886637 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6969ad-7e82-4fe2-9080-741c4ad4371b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b692edd3e8c41289cc42e1afb4a6dd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:22:06.887634 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6969ad-7e82-4fe2-9080-741c4ad4371b service placement] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:22:06.888135 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6969ad-7e82-4fe2-9080-741c4ad4371b service placement] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:22:06.888625 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6969ad-7e82-4fe2-9080-741c4ad4371b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:22:06.891021 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-6b6969ad-7e82-4fe2-9080-741c4ad4371b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:22:06.908290 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6969ad-7e82-4fe2-9080-741c4ad4371b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:22:06.910239 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6969ad-7e82-4fe2-9080-741c4ad4371b service placement] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:22:06.935335 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ed2e63-3b6e-48c3-a84b-b357d3e18a7a admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:22:06.942192 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b69df1e-6393-4f39-86da-2dbef26b331e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:22:06.943974 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b69df1e-6393-4f39-86da-2dbef26b331e admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:22:06.944244 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1155/2323] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:22:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:22:06.944657 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b69df1e-6393-4f39-86da-2dbef26b331e admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:22:06.945112 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b69df1e-6393-4f39-86da-2dbef26b331e admin admin] PATH_INFO: `/v3/projects/6e0216ac96ee427a96843e8e0fcb5a23/users/5a079cb15fe7498e9dbb5df7804643d5/roles/62f19ea272b447ccaab1576cd9bf4fd9` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:22:06.945726 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7545ac59-4c30-40e1-bd14-16d79bb983b3 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:22:06.946019 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7545ac59-4c30-40e1-bd14-16d79bb983b3 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:22:06.946313 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7545ac59-4c30-40e1-bd14-16d79bb983b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:22:06.973212 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b69df1e-6393-4f39-86da-2dbef26b331e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e0216ac96ee427a96843e8e0fcb5a23, user_id=5a079cb15fe7498e9dbb5df7804643d5, role_id=62f19ea272b447ccaab1576cd9bf4fd9)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:22:06.979988 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b69df1e-6393-4f39-86da-2dbef26b331e admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:22:06.998765 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-7545ac59-4c30-40e1-bd14-16d79bb983b3 None None] MFA Rules not processed for user `97897f84a40f4401929fe2746945b6e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:22:07.000262 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-7545ac59-4c30-40e1-bd14-16d79bb983b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:22:07.020327 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:22:07.051034 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1169/2324] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:22:06 2022] PUT /identity/v3/projects/6e0216ac96ee427a96843e8e0fcb5a23/users/5a079cb15fe7498e9dbb5df7804643d5/roles/62f19ea272b447ccaab1576cd9bf4fd9 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:22:07.054085 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbfaae6-cb2a-4c86-8035-118b9f6e76e1 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:22:07.056224 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbfaae6-cb2a-4c86-8035-118b9f6e76e1 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:22:07.056361 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbfaae6-cb2a-4c86-8035-118b9f6e76e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:22:07.087735 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-6bbfaae6-cb2a-4c86-8035-118b9f6e76e1 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:22:07.089464 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-6bbfaae6-cb2a-4c86-8035-118b9f6e76e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:22:07.096481 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1156/2325] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:22:06 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:22:07.097589 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7545ac59-4c30-40e1-bd14-16d79bb983b3 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:22:07.134237 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7545ac59-4c30-40e1-bd14-16d79bb983b3 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:22:07.223657 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1170/2326] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:22:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:22:07.224950 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbfaae6-cb2a-4c86-8035-118b9f6e76e1 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:22:07.226448 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-6bbfaae6-cb2a-4c86-8035-118b9f6e76e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:22:07.247169 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e4816e-9049-4614-8054-7a1ccdc01ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:22:07.250866 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e4816e-9049-4614-8054-7a1ccdc01ea7 admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:22:07.250866 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e4816e-9049-4614-8054-7a1ccdc01ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:22:07.250866 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e4816e-9049-4614-8054-7a1ccdc01ea7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:22:07.251774 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e4816e-9049-4614-8054-7a1ccdc01ea7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:22:07.255733 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e4816e-9049-4614-8054-7a1ccdc01ea7 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:22:07.284695 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1157/2327] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:22:07 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:22:07.286641 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8a05c5-f13e-4040-976a-1f2af6771a46 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:22:07.287218 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8a05c5-f13e-4040-976a-1f2af6771a46 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:22:07.287699 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8a05c5-f13e-4040-976a-1f2af6771a46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:22:07.325883 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-1c8a05c5-f13e-4040-976a-1f2af6771a46 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:22:07.327968 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-1c8a05c5-f13e-4040-976a-1f2af6771a46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:22:07.341648 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1158/2328] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:22:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:22:07.343294 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8a05c5-f13e-4040-976a-1f2af6771a46 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:22:07.358469 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8a05c5-f13e-4040-976a-1f2af6771a46 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:22:07.369499 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542ee0a5-bec3-4b1a-9683-48ac3607269a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:22:07.370481 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ee0a5-bec3-4b1a-9683-48ac3607269a admin admin] REQUEST_METHOD: `PUT` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:22:07.370882 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ee0a5-bec3-4b1a-9683-48ac3607269a admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:22:07.374768 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ee0a5-bec3-4b1a-9683-48ac3607269a admin admin] PATH_INFO: `/v3/projects/6e0216ac96ee427a96843e8e0fcb5a23/users/5a079cb15fe7498e9dbb5df7804643d5/roles/62f19ea272b447ccaab1576cd9bf4fd9` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:22:07.379112 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542ee0a5-bec3-4b1a-9683-48ac3607269a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e0216ac96ee427a96843e8e0fcb5a23, user_id=5a079cb15fe7498e9dbb5df7804643d5, role_id=62f19ea272b447ccaab1576cd9bf4fd9)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:22:07.383291 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542ee0a5-bec3-4b1a-9683-48ac3607269a admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:22:07.388000 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbfaae6-cb2a-4c86-8035-118b9f6e76e1 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:22:07.397605 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1159/2329] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:22:07 2022] PUT /identity/v3/projects/6e0216ac96ee427a96843e8e0fcb5a23/users/5a079cb15fe7498e9dbb5df7804643d5/roles/62f19ea272b447ccaab1576cd9bf4fd9 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:22:07.398550 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542ee0a5-bec3-4b1a-9683-48ac3607269a admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:22:07.412519 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed6fd19-04ef-4c56-b218-b50edbefa6b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cd171bf4df94f8fab397e42835feeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:22:07.413480 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed6fd19-04ef-4c56-b218-b50edbefa6b0 service nova] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:22:07.413877 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed6fd19-04ef-4c56-b218-b50edbefa6b0 service nova] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:22:07.414295 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed6fd19-04ef-4c56-b218-b50edbefa6b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:22:07.418531 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-fed6fd19-04ef-4c56-b218-b50edbefa6b0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:22:07.517040 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542ee0a5-bec3-4b1a-9683-48ac3607269a admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:22:07.524623 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941d660e-b20d-4577-b6fd-1b6ed0b06e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:22:07.525823 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941d660e-b20d-4577-b6fd-1b6ed0b06e85 admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:22:07.527090 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941d660e-b20d-4577-b6fd-1b6ed0b06e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:22:07.527324 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941d660e-b20d-4577-b6fd-1b6ed0b06e85 admin admin] PATH_INFO: `/v3/domains` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:22:07.528477 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941d660e-b20d-4577-b6fd-1b6ed0b06e85 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:22:07.528773 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941d660e-b20d-4577-b6fd-1b6ed0b06e85 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:22:07.530662 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941d660e-b20d-4577-b6fd-1b6ed0b06e85 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:22:07.544284 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:22:07.557349 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed6fd19-04ef-4c56-b218-b50edbefa6b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:22:07.559024 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed6fd19-04ef-4c56-b218-b50edbefa6b0 service nova] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:22:07.570651 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1160/2330] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Aug 25 03:22:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:22:07.571747 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4607a4d4-cff1-4418-99a6-760aaacfc308 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:22:07.572039 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4607a4d4-cff1-4418-99a6-760aaacfc308 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:22:07.572313 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4607a4d4-cff1-4418-99a6-760aaacfc308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:22:07.578891 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1171/2331] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:22:07 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:22:07.580478 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed6fd19-04ef-4c56-b218-b50edbefa6b0 service nova] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:22:07.592612 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed6fd19-04ef-4c56-b218-b50edbefa6b0 service nova] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:22:07.606457 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bfdc28-cedb-43b8-9b22-9c96bfdafb42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:22:07.607389 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bfdc28-cedb-43b8-9b22-9c96bfdafb42 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:22:07.608180 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bfdc28-cedb-43b8-9b22-9c96bfdafb42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:22:07.608696 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bfdc28-cedb-43b8-9b22-9c96bfdafb42 admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:22:07.609603 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bfdc28-cedb-43b8-9b22-9c96bfdafb42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:22:07.612542 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bfdc28-cedb-43b8-9b22-9c96bfdafb42 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:22:07.704528 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-4607a4d4-cff1-4418-99a6-760aaacfc308 None None] MFA Rules not processed for user `5a079cb15fe7498e9dbb5df7804643d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:22:07.707061 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-4607a4d4-cff1-4418-99a6-760aaacfc308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:22:07.707212 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1172/2332] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:22:07 2022] POST /identity/v3/projects => generated 389 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:22:07.712342 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bfdc28-cedb-43b8-9b22-9c96bfdafb42 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:22:07.731660 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:22:07.859879 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bfdc28-cedb-43b8-9b22-9c96bfdafb42 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:22:07.875858 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c935bb38-9899-4210-bee1-f27cf7beacdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:22:07.876928 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c935bb38-9899-4210-bee1-f27cf7beacdc admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:22:07.877218 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c935bb38-9899-4210-bee1-f27cf7beacdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:22:07.877561 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c935bb38-9899-4210-bee1-f27cf7beacdc admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:22:07.878315 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c935bb38-9899-4210-bee1-f27cf7beacdc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:22:07.883029 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c935bb38-9899-4210-bee1-f27cf7beacdc admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:22:07.920142 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1161/2333] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:22:07 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 349 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:22:07.935509 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4607a4d4-cff1-4418-99a6-760aaacfc308 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:22:07.937158 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-4607a4d4-cff1-4418-99a6-760aaacfc308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:22:08.003133 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1173/2334] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:22:07 2022] POST /identity/v3/users => generated 295 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:22:08.004131 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c935bb38-9899-4210-bee1-f27cf7beacdc admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:22:08.028576 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4607a4d4-cff1-4418-99a6-760aaacfc308 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:22:08.033100 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec670ffa-c9e9-4314-9e12-d04725e73b7d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60193ba9e1c14df3bc9eebcc6f28f2d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:22:08.033702 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c935bb38-9899-4210-bee1-f27cf7beacdc admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:22:08.033825 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec670ffa-c9e9-4314-9e12-d04725e73b7d service cinder] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:22:08.033977 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec670ffa-c9e9-4314-9e12-d04725e73b7d service cinder] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:22:08.034291 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec670ffa-c9e9-4314-9e12-d04725e73b7d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:22:08.037456 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-ec670ffa-c9e9-4314-9e12-d04725e73b7d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:22:08.049934 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91f37ae-6121-4a30-a543-64342df8c9d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:22:08.049934 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91f37ae-6121-4a30-a543-64342df8c9d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:22:08.049934 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91f37ae-6121-4a30-a543-64342df8c9d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:22:08.050235 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91f37ae-6121-4a30-a543-64342df8c9d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:22:08.050861 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91f37ae-6121-4a30-a543-64342df8c9d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:22:08.052513 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91f37ae-6121-4a30-a543-64342df8c9d8 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:22:08.066651 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1174/2335] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:22:08 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:22:08.070978 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91f37ae-6121-4a30-a543-64342df8c9d8 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:22:08.088157 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91f37ae-6121-4a30-a543-64342df8c9d8 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:22:08.098070 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173a72e6-0be2-4308-8304-ead13a035441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:22:08.099742 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173a72e6-0be2-4308-8304-ead13a035441 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:22:08.101219 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173a72e6-0be2-4308-8304-ead13a035441 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:22:08.101661 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173a72e6-0be2-4308-8304-ead13a035441 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:22:08.102470 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173a72e6-0be2-4308-8304-ead13a035441 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:22:08.104264 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173a72e6-0be2-4308-8304-ead13a035441 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:22:08.119330 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1175/2336] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:22:08 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:22:08.123032 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173a72e6-0be2-4308-8304-ead13a035441 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:22:08.131645 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec670ffa-c9e9-4314-9e12-d04725e73b7d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:22:08.133287 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec670ffa-c9e9-4314-9e12-d04725e73b7d service cinder] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:22:08.142701 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173a72e6-0be2-4308-8304-ead13a035441 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:22:08.145305 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1162/2337] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:22:07 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:22:08.148375 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa886cf-7b4a-4860-a0e5-d2886f809615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:22:08.149231 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa886cf-7b4a-4860-a0e5-d2886f809615 admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:22:08.149848 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa886cf-7b4a-4860-a0e5-d2886f809615 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:22:08.150472 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa886cf-7b4a-4860-a0e5-d2886f809615 admin admin] PATH_INFO: `/v3/projects/dd784a04c8534d4bbd596a1c45a52d31/users/ff639b85df1d4127a7ec3e9c716224c5/roles/3255f77192524a16bc405173106871fc` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:22:08.169394 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa886cf-7b4a-4860-a0e5-d2886f809615 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd784a04c8534d4bbd596a1c45a52d31, user_id=ff639b85df1d4127a7ec3e9c716224c5, role_id=3255f77192524a16bc405173106871fc)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:22:08.174088 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa886cf-7b4a-4860-a0e5-d2886f809615 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:22:08.478374 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1176/2338] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:22:08 2022] PUT /identity/v3/projects/dd784a04c8534d4bbd596a1c45a52d31/users/ff639b85df1d4127a7ec3e9c716224c5/roles/3255f77192524a16bc405173106871fc => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:22:08.478714 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a9bbb1-6ef6-448a-bb71-9b984d6e8ff5 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:22:08.478714 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a9bbb1-6ef6-448a-bb71-9b984d6e8ff5 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:22:08.478714 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a9bbb1-6ef6-448a-bb71-9b984d6e8ff5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:22:08.550252 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-d0a9bbb1-6ef6-448a-bb71-9b984d6e8ff5 None None] MFA Rules not processed for user `ff639b85df1d4127a7ec3e9c716224c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:22:08.553433 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-d0a9bbb1-6ef6-448a-bb71-9b984d6e8ff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:22:08.659325 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:22:08.723175 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1163/2339] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:22:08 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:22:08.736162 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa886cf-7b4a-4860-a0e5-d2886f809615 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:22:08.738348 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-2fa886cf-7b4a-4860-a0e5-d2886f809615 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:22:08.824669 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa886cf-7b4a-4860-a0e5-d2886f809615 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:22:08.832177 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5be8d3-97e1-4cfb-9cf2-b0469c75e511 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60193ba9e1c14df3bc9eebcc6f28f2d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:22:08.832177 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5be8d3-97e1-4cfb-9cf2-b0469c75e511 service cinder] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:22:08.832177 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5be8d3-97e1-4cfb-9cf2-b0469c75e511 service cinder] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:22:08.832177 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5be8d3-97e1-4cfb-9cf2-b0469c75e511 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:22:08.840696 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5be8d3-97e1-4cfb-9cf2-b0469c75e511 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:22:08.842461 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5be8d3-97e1-4cfb-9cf2-b0469c75e511 service cinder] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:22:08.882349 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1177/2340] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:22:08 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:22:13.005465 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a9bbb1-6ef6-448a-bb71-9b984d6e8ff5 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:22:13.147167 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a9bbb1-6ef6-448a-bb71-9b984d6e8ff5 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:22:13.152228 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcf095e-7653-4742-9f94-c79e08c90412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:22:13.152868 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcf095e-7653-4742-9f94-c79e08c90412 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:22:13.153302 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcf095e-7653-4742-9f94-c79e08c90412 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:22:13.153830 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcf095e-7653-4742-9f94-c79e08c90412 admin admin] PATH_INFO: `/v3/users/fe34364c80b24b3698593cb2ca9ed1e5` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:22:13.155131 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcf095e-7653-4742-9f94-c79e08c90412 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe34364c80b24b3698593cb2ca9ed1e5)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:22:13.157441 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcf095e-7653-4742-9f94-c79e08c90412 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:22:13.412329 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-edcf095e-7653-4742-9f94-c79e08c90412 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe34364c80b24b3698593cb2ca9ed1e5'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:22:13.466554 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-edcf095e-7653-4742-9f94-c79e08c90412 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe34364c80b24b3698593cb2ca9ed1e5'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:22:13.467599 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-edcf095e-7653-4742-9f94-c79e08c90412 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe34364c80b24b3698593cb2ca9ed1e5'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:22:13.468235 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-edcf095e-7653-4742-9f94-c79e08c90412 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe34364c80b24b3698593cb2ca9ed1e5'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:22:13.488874 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-edcf095e-7653-4742-9f94-c79e08c90412 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe34364c80b24b3698593cb2ca9ed1e5'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:22:13.504086 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1164/2341] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:22:13 2022] DELETE /identity/v3/users/fe34364c80b24b3698593cb2ca9ed1e5 => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:22:13.509646 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a4bf44-0768-4079-826d-6778079c0b3c None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:22:13.510176 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a4bf44-0768-4079-826d-6778079c0b3c None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:22:13.510176 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a4bf44-0768-4079-826d-6778079c0b3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:22:13.536975 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-d4a4bf44-0768-4079-826d-6778079c0b3c None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:22:13.538248 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-d4a4bf44-0768-4079-826d-6778079c0b3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:22:13.556273 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:22:13.633300 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1178/2342] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:22:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:22:13.644044 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcf095e-7653-4742-9f94-c79e08c90412 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:22:13.645611 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-edcf095e-7653-4742-9f94-c79e08c90412 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:22:13.735465 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcf095e-7653-4742-9f94-c79e08c90412 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:22:13.742120 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6647d36e-9982-432b-af75-d8d68d66c598 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:22:13.742986 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6647d36e-9982-432b-af75-d8d68d66c598 service neutron] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:22:13.743367 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6647d36e-9982-432b-af75-d8d68d66c598 service neutron] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:22:13.743783 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6647d36e-9982-432b-af75-d8d68d66c598 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:22:13.755897 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6647d36e-9982-432b-af75-d8d68d66c598 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:22:13.757441 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6647d36e-9982-432b-af75-d8d68d66c598 service neutron] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:22:13.769568 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1165/2343] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:22:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:22:14.325081 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a4bf44-0768-4079-826d-6778079c0b3c None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:22:14.326925 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-d4a4bf44-0768-4079-826d-6778079c0b3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:22:14.344118 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a4bf44-0768-4079-826d-6778079c0b3c None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:22:14.372055 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b37f398-abb6-4507-a978-5a587edbb331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:22:14.373341 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b37f398-abb6-4507-a978-5a587edbb331 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:22:14.373895 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b37f398-abb6-4507-a978-5a587edbb331 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:22:14.374347 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b37f398-abb6-4507-a978-5a587edbb331 admin admin] PATH_INFO: `/v3/projects/b133e68717b94a428ea3d78e3aaee0fb` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:22:14.375825 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b37f398-abb6-4507-a978-5a587edbb331 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b133e68717b94a428ea3d78e3aaee0fb)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:22:14.379464 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b37f398-abb6-4507-a978-5a587edbb331 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:22:14.513367 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-9b37f398-abb6-4507-a978-5a587edbb331 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b133e68717b94a428ea3d78e3aaee0fb', 'request_id': 'req-9b37f398-abb6-4507-a978-5a587edbb331'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:22:14.557716 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-9b37f398-abb6-4507-a978-5a587edbb331 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b133e68717b94a428ea3d78e3aaee0fb', 'request_id': 'req-9b37f398-abb6-4507-a978-5a587edbb331'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:22:14.565247 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-9b37f398-abb6-4507-a978-5a587edbb331 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b133e68717b94a428ea3d78e3aaee0fb', 'request_id': 'req-9b37f398-abb6-4507-a978-5a587edbb331'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:22:14.583816 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-9b37f398-abb6-4507-a978-5a587edbb331 admin admin] The token cache is being invalidate because project b133e68717b94a428ea3d78e3aaee0fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:22:14.584872 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-9b37f398-abb6-4507-a978-5a587edbb331 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:22:14.585945 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-9b37f398-abb6-4507-a978-5a587edbb331 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:22:14.588411 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1179/2344] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:22:14 2022] DELETE /identity/v3/projects/b133e68717b94a428ea3d78e3aaee0fb => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:22:14.592398 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6647d36e-9982-432b-af75-d8d68d66c598 service neutron] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:22:14.603154 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6647d36e-9982-432b-af75-d8d68d66c598 service neutron] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:22:14.616037 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:22:14.668380 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac1a914-fa14-4e40-8e0b-942fc8d59bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:22:14.671288 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac1a914-fa14-4e40-8e0b-942fc8d59bae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:22:14.671781 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac1a914-fa14-4e40-8e0b-942fc8d59bae admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:22:14.671961 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac1a914-fa14-4e40-8e0b-942fc8d59bae admin admin] PATH_INFO: `/v3/users/5a079cb15fe7498e9dbb5df7804643d5` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:22:14.674086 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac1a914-fa14-4e40-8e0b-942fc8d59bae admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a079cb15fe7498e9dbb5df7804643d5)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:22:14.676370 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac1a914-fa14-4e40-8e0b-942fc8d59bae admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:22:14.885789 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-cac1a914-fa14-4e40-8e0b-942fc8d59bae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a079cb15fe7498e9dbb5df7804643d5'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:22:14.952682 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-cac1a914-fa14-4e40-8e0b-942fc8d59bae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a079cb15fe7498e9dbb5df7804643d5'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:22:14.953332 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-cac1a914-fa14-4e40-8e0b-942fc8d59bae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a079cb15fe7498e9dbb5df7804643d5'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:22:14.953951 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-cac1a914-fa14-4e40-8e0b-942fc8d59bae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a079cb15fe7498e9dbb5df7804643d5'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:22:14.974588 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-cac1a914-fa14-4e40-8e0b-942fc8d59bae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a079cb15fe7498e9dbb5df7804643d5'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:22:14.994520 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1166/2345] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:22:14 2022] DELETE /identity/v3/users/5a079cb15fe7498e9dbb5df7804643d5 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:22:15.428795 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b37f398-abb6-4507-a978-5a587edbb331 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:22:15.430273 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-9b37f398-abb6-4507-a978-5a587edbb331 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:22:15.461088 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b37f398-abb6-4507-a978-5a587edbb331 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:22:15.544681 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a707a83-2838-4340-8ceb-a4b7739c25e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:22:15.546169 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a707a83-2838-4340-8ceb-a4b7739c25e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:22:15.546837 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a707a83-2838-4340-8ceb-a4b7739c25e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:22:15.547593 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a707a83-2838-4340-8ceb-a4b7739c25e9 admin admin] PATH_INFO: `/v3/projects/6e0216ac96ee427a96843e8e0fcb5a23` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:22:15.549712 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a707a83-2838-4340-8ceb-a4b7739c25e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e0216ac96ee427a96843e8e0fcb5a23)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:22:15.552450 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a707a83-2838-4340-8ceb-a4b7739c25e9 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:22:15.712071 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:22:15.715699 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-4a707a83-2838-4340-8ceb-a4b7739c25e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e0216ac96ee427a96843e8e0fcb5a23', 'request_id': 'req-4a707a83-2838-4340-8ceb-a4b7739c25e9'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:22:15.787227 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-4a707a83-2838-4340-8ceb-a4b7739c25e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e0216ac96ee427a96843e8e0fcb5a23', 'request_id': 'req-4a707a83-2838-4340-8ceb-a4b7739c25e9'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:22:15.795374 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-4a707a83-2838-4340-8ceb-a4b7739c25e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e0216ac96ee427a96843e8e0fcb5a23', 'request_id': 'req-4a707a83-2838-4340-8ceb-a4b7739c25e9'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:22:15.812132 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-4a707a83-2838-4340-8ceb-a4b7739c25e9 admin admin] The token cache is being invalidate because project 6e0216ac96ee427a96843e8e0fcb5a23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:22:15.812634 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-4a707a83-2838-4340-8ceb-a4b7739c25e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:22:15.813217 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-4a707a83-2838-4340-8ceb-a4b7739c25e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:22:15.820122 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1180/2346] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:22:15 2022] DELETE /identity/v3/projects/6e0216ac96ee427a96843e8e0fcb5a23 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:22:15.821832 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ce6b09-430b-485a-a686-7f21303027a3 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:22:15.822594 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ce6b09-430b-485a-a686-7f21303027a3 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:22:15.822790 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ce6b09-430b-485a-a686-7f21303027a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:22:15.848352 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-d7ce6b09-430b-485a-a686-7f21303027a3 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:22:15.849857 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-d7ce6b09-430b-485a-a686-7f21303027a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:22:15.937963 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1167/2347] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:22:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:22:15.939508 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8474fe1c-74bc-4cae-9849-f18fa3fb85ea None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:22:15.939847 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8474fe1c-74bc-4cae-9849-f18fa3fb85ea None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:22:15.940124 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8474fe1c-74bc-4cae-9849-f18fa3fb85ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:22:15.976299 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-8474fe1c-74bc-4cae-9849-f18fa3fb85ea None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:22:15.977600 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-8474fe1c-74bc-4cae-9849-f18fa3fb85ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:22:16.002997 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1181/2348] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:22:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:22:16.008784 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ce6b09-430b-485a-a686-7f21303027a3 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:22:16.024438 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ce6b09-430b-485a-a686-7f21303027a3 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:22:16.037944 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84acdc37-bdd2-4329-906d-6206235d23ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:22:16.038618 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84acdc37-bdd2-4329-906d-6206235d23ba admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:22:16.038914 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84acdc37-bdd2-4329-906d-6206235d23ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:22:16.039284 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84acdc37-bdd2-4329-906d-6206235d23ba admin admin] PATH_INFO: `/v3/domains` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:22:16.040149 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84acdc37-bdd2-4329-906d-6206235d23ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:22:16.040525 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84acdc37-bdd2-4329-906d-6206235d23ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:22:16.042749 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84acdc37-bdd2-4329-906d-6206235d23ba admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:22:16.059562 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1168/2349] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Aug 25 03:22:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:22:16.063810 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8474fe1c-74bc-4cae-9849-f18fa3fb85ea None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:22:16.076151 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8474fe1c-74bc-4cae-9849-f18fa3fb85ea None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:22:16.082053 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5d9092-1d49-4c4a-a3de-8b6ad8692a6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:22:16.082854 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5d9092-1d49-4c4a-a3de-8b6ad8692a6f admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:22:16.083419 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5d9092-1d49-4c4a-a3de-8b6ad8692a6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:22:16.083710 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5d9092-1d49-4c4a-a3de-8b6ad8692a6f admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:22:16.084429 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5d9092-1d49-4c4a-a3de-8b6ad8692a6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:22:16.086547 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5d9092-1d49-4c4a-a3de-8b6ad8692a6f admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:22:16.171433 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1182/2350] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:22:16 2022] POST /identity/v3/projects => generated 377 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:22:16.175605 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84acdc37-bdd2-4329-906d-6206235d23ba admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:22:16.188506 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84acdc37-bdd2-4329-906d-6206235d23ba admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:22:16.213626 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:22:16.257241 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3607e3c3-0d31-4b18-b0cf-5962d1dfb0c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:22:16.257939 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3607e3c3-0d31-4b18-b0cf-5962d1dfb0c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:22:16.258439 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3607e3c3-0d31-4b18-b0cf-5962d1dfb0c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:22:16.258724 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3607e3c3-0d31-4b18-b0cf-5962d1dfb0c6 admin admin] PATH_INFO: `/v3/users` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:22:16.260264 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3607e3c3-0d31-4b18-b0cf-5962d1dfb0c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:22:16.262800 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3607e3c3-0d31-4b18-b0cf-5962d1dfb0c6 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:22:16.436733 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1169/2351] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:22:16 2022] POST /identity/v3/users => generated 289 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:22:16.441349 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5d9092-1d49-4c4a-a3de-8b6ad8692a6f admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:22:16.454439 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5d9092-1d49-4c4a-a3de-8b6ad8692a6f admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:22:16.474583 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d83164-1424-43bf-ae40-2094b2879ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:22:16.474583 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d83164-1424-43bf-ae40-2094b2879ca3 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:22:16.474583 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d83164-1424-43bf-ae40-2094b2879ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:22:16.474583 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d83164-1424-43bf-ae40-2094b2879ca3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:22:16.475076 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d83164-1424-43bf-ae40-2094b2879ca3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:22:16.475076 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d83164-1424-43bf-ae40-2094b2879ca3 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:22:16.491408 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1183/2352] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:22:16 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 25 03:22:16.492627 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3607e3c3-0d31-4b18-b0cf-5962d1dfb0c6 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:22:16.511844 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3607e3c3-0d31-4b18-b0cf-5962d1dfb0c6 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:22:16.541996 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd3bcfd-ac09-4db7-b711-daf2562f1e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:22:16.554474 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd3bcfd-ac09-4db7-b711-daf2562f1e2c admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:22:16.554896 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd3bcfd-ac09-4db7-b711-daf2562f1e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:22:16.556203 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd3bcfd-ac09-4db7-b711-daf2562f1e2c admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:22:16.556958 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd3bcfd-ac09-4db7-b711-daf2562f1e2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:22:16.558904 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd3bcfd-ac09-4db7-b711-daf2562f1e2c admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:22:16.574526 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1170/2353] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:22:16 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 25 03:22:16.576262 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d83164-1424-43bf-ae40-2094b2879ca3 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:22:16.590371 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d83164-1424-43bf-ae40-2094b2879ca3 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:22:16.595508 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82743ca4-e12a-4000-942d-81b6fc90115e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:22:16.596163 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82743ca4-e12a-4000-942d-81b6fc90115e admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:22:16.596439 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82743ca4-e12a-4000-942d-81b6fc90115e admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:22:16.596714 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82743ca4-e12a-4000-942d-81b6fc90115e admin admin] PATH_INFO: `/v3/projects/2ccf846bb93a4e7d87054ec280216c84/users/832a8721625f40759baf0044867664cd/roles/3255f77192524a16bc405173106871fc` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:22:16.612128 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82743ca4-e12a-4000-942d-81b6fc90115e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ccf846bb93a4e7d87054ec280216c84, user_id=832a8721625f40759baf0044867664cd, role_id=3255f77192524a16bc405173106871fc)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:22:16.615441 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82743ca4-e12a-4000-942d-81b6fc90115e admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:22:16.748551 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1184/2354] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:22:16 2022] PUT /identity/v3/projects/2ccf846bb93a4e7d87054ec280216c84/users/832a8721625f40759baf0044867664cd/roles/3255f77192524a16bc405173106871fc => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:22:16.761689 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62a071d-6ac8-4aab-a2cb-78a3a7c055ed None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:22:16.762260 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62a071d-6ac8-4aab-a2cb-78a3a7c055ed None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:22:16.762559 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62a071d-6ac8-4aab-a2cb-78a3a7c055ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:22:16.799577 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-f62a071d-6ac8-4aab-a2cb-78a3a7c055ed None None] MFA Rules not processed for user `832a8721625f40759baf0044867664cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:22:16.801673 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-f62a071d-6ac8-4aab-a2cb-78a3a7c055ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:22:16.894788 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1171/2355] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:22:16 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:22:16.908677 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82743ca4-e12a-4000-942d-81b6fc90115e admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:22:16.910647 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-82743ca4-e12a-4000-942d-81b6fc90115e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:22:16.993379 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82743ca4-e12a-4000-942d-81b6fc90115e admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:22:16.999368 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7210baf7-656a-4bbe-b866-5bc7909b9c86 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60193ba9e1c14df3bc9eebcc6f28f2d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:22:17.001180 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7210baf7-656a-4bbe-b866-5bc7909b9c86 service cinder] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:22:17.001180 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7210baf7-656a-4bbe-b866-5bc7909b9c86 service cinder] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:22:17.001180 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7210baf7-656a-4bbe-b866-5bc7909b9c86 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:22:17.010647 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7210baf7-656a-4bbe-b866-5bc7909b9c86 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:22:17.012076 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7210baf7-656a-4bbe-b866-5bc7909b9c86 service cinder] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:22:17.022844 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1185/2356] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:22:16 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:22:26.892401 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=57,keystone:INSERT=3 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:22:27.017276 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=61,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:22:37.528418 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7d9493-eb96-4a15-8002-0f1358ecd8e5 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:22:37.528876 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7d9493-eb96-4a15-8002-0f1358ecd8e5 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:22:37.528973 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7d9493-eb96-4a15-8002-0f1358ecd8e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:22:37.548188 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-da7d9493-eb96-4a15-8002-0f1358ecd8e5 None None] MFA Rules not processed for user `712c35b765b64919aa3eaaf02892590d`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:22:37.549467 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-da7d9493-eb96-4a15-8002-0f1358ecd8e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:22:37.605114 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1172/2357] 10.222.0.58 () {58 vars in 967 bytes} [Thu Aug 25 03:22:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:22:37.618056 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7210baf7-656a-4bbe-b866-5bc7909b9c86 service cinder] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:22:37.619604 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-7210baf7-656a-4bbe-b866-5bc7909b9c86 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:22:37.698651 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7210baf7-656a-4bbe-b866-5bc7909b9c86 service cinder] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:22:37.704385 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b99a0af-f7cb-464f-940b-58310312cd6a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5047e1a7cd2d416d90b2d8c5a6596522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:22:37.704530 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b99a0af-f7cb-464f-940b-58310312cd6a service swift] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:22:37.704791 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b99a0af-f7cb-464f-940b-58310312cd6a service swift] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:22:37.705059 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b99a0af-f7cb-464f-940b-58310312cd6a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:22:37.718713 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b99a0af-f7cb-464f-940b-58310312cd6a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:22:37.721431 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b99a0af-f7cb-464f-940b-58310312cd6a service swift] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:22:37.732423 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1186/2358] 10.222.0.58 () {60 vars in 1384 bytes} [Thu Aug 25 03:22:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 25 03:22:47.597987 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=25 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:22:47.728709 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=32 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:22:51.506463 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7d9493-eb96-4a15-8002-0f1358ecd8e5 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:22:51.508562 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-da7d9493-eb96-4a15-8002-0f1358ecd8e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:22:51.593079 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7d9493-eb96-4a15-8002-0f1358ecd8e5 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:22:51.615691 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5d82f8-aaa7-4532-b77b-6def9bd6d232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:22:51.615691 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5d82f8-aaa7-4532-b77b-6def9bd6d232 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:22:51.615691 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5d82f8-aaa7-4532-b77b-6def9bd6d232 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:22:51.615691 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5d82f8-aaa7-4532-b77b-6def9bd6d232 admin admin] PATH_INFO: `/v3/users/2552825edca0418392f877ea9488e36f` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:22:51.618835 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5d82f8-aaa7-4532-b77b-6def9bd6d232 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2552825edca0418392f877ea9488e36f)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:22:51.621316 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5d82f8-aaa7-4532-b77b-6def9bd6d232 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:22:51.796392 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-4c5d82f8-aaa7-4532-b77b-6def9bd6d232 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2552825edca0418392f877ea9488e36f'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:22:51.841606 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-4c5d82f8-aaa7-4532-b77b-6def9bd6d232 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2552825edca0418392f877ea9488e36f'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:22:51.842406 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-4c5d82f8-aaa7-4532-b77b-6def9bd6d232 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2552825edca0418392f877ea9488e36f'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:22:51.843072 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-4c5d82f8-aaa7-4532-b77b-6def9bd6d232 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2552825edca0418392f877ea9488e36f'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:22:51.859464 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-4c5d82f8-aaa7-4532-b77b-6def9bd6d232 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2552825edca0418392f877ea9488e36f'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:22:51.872701 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1173/2359] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:22:51 2022] DELETE /identity/v3/users/2552825edca0418392f877ea9488e36f => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:22:52.148730 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b99a0af-f7cb-464f-940b-58310312cd6a service swift] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:22:52.170916 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b99a0af-f7cb-464f-940b-58310312cd6a service swift] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:22:52.265691 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142be44a-9d13-407e-965c-cb35fe3aa481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:22:52.266367 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142be44a-9d13-407e-965c-cb35fe3aa481 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:22:52.266649 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142be44a-9d13-407e-965c-cb35fe3aa481 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:22:52.266925 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142be44a-9d13-407e-965c-cb35fe3aa481 admin admin] PATH_INFO: `/v3/projects/882db843d4644ac29365868d1deb9a27` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:22:52.268082 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142be44a-9d13-407e-965c-cb35fe3aa481 admin admin] RBAC: Authorizing `identity:delete_project(project_id=882db843d4644ac29365868d1deb9a27)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:22:52.270004 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142be44a-9d13-407e-965c-cb35fe3aa481 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:22:52.389157 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-142be44a-9d13-407e-965c-cb35fe3aa481 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '882db843d4644ac29365868d1deb9a27', 'request_id': 'req-142be44a-9d13-407e-965c-cb35fe3aa481'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:22:52.448399 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-142be44a-9d13-407e-965c-cb35fe3aa481 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '882db843d4644ac29365868d1deb9a27', 'request_id': 'req-142be44a-9d13-407e-965c-cb35fe3aa481'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:22:52.455440 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-142be44a-9d13-407e-965c-cb35fe3aa481 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '882db843d4644ac29365868d1deb9a27', 'request_id': 'req-142be44a-9d13-407e-965c-cb35fe3aa481'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:22:52.468321 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-142be44a-9d13-407e-965c-cb35fe3aa481 admin admin] The token cache is being invalidate because project 882db843d4644ac29365868d1deb9a27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:22:52.468817 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-142be44a-9d13-407e-965c-cb35fe3aa481 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:22:52.469811 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-142be44a-9d13-407e-965c-cb35fe3aa481 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:22:52.471683 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1187/2360] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:22:52 2022] DELETE /identity/v3/projects/882db843d4644ac29365868d1deb9a27 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:22:52.477113 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9ede11-740d-47e8-b12f-4e8137af5412 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:22:52.477404 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9ede11-740d-47e8-b12f-4e8137af5412 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:22:52.477683 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9ede11-740d-47e8-b12f-4e8137af5412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:22:52.502680 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-8c9ede11-740d-47e8-b12f-4e8137af5412 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:22:52.503901 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-8c9ede11-740d-47e8-b12f-4e8137af5412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:22:52.569120 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1174/2361] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:22:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:22:52.578616 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8708281e-1d56-482a-a161-f38feb845781 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:22:52.578945 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8708281e-1d56-482a-a161-f38feb845781 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:22:52.579191 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8708281e-1d56-482a-a161-f38feb845781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:22:52.606182 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-8708281e-1d56-482a-a161-f38feb845781 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:22:52.607457 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-8708281e-1d56-482a-a161-f38feb845781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:22:52.615607 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1188/2362] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:22:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:22:52.619949 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9ede11-740d-47e8-b12f-4e8137af5412 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:22:52.631233 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9ede11-740d-47e8-b12f-4e8137af5412 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:22:52.637537 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794e0be5-52c0-48b0-af2f-06ca9b9a0226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:22:52.637712 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794e0be5-52c0-48b0-af2f-06ca9b9a0226 admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:22:52.639507 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794e0be5-52c0-48b0-af2f-06ca9b9a0226 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:22:52.639507 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794e0be5-52c0-48b0-af2f-06ca9b9a0226 admin admin] PATH_INFO: `/v3/domains` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:22:52.639507 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794e0be5-52c0-48b0-af2f-06ca9b9a0226 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:22:52.639753 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794e0be5-52c0-48b0-af2f-06ca9b9a0226 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:22:52.641480 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794e0be5-52c0-48b0-af2f-06ca9b9a0226 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:22:52.663463 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1175/2363] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Aug 25 03:22:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:22:52.666827 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8708281e-1d56-482a-a161-f38feb845781 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:22:52.677912 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8708281e-1d56-482a-a161-f38feb845781 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:22:52.684552 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6283632e-943a-4426-973e-8a88c532c064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:22:52.685180 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6283632e-943a-4426-973e-8a88c532c064 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:22:52.685494 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6283632e-943a-4426-973e-8a88c532c064 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:22:52.685754 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6283632e-943a-4426-973e-8a88c532c064 admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:22:52.687101 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6283632e-943a-4426-973e-8a88c532c064 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:22:52.692128 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6283632e-943a-4426-973e-8a88c532c064 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:22:52.795872 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1189/2364] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:22:52 2022] POST /identity/v3/projects => generated 363 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:22:52.797235 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794e0be5-52c0-48b0-af2f-06ca9b9a0226 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:22:52.808502 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794e0be5-52c0-48b0-af2f-06ca9b9a0226 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:22:52.812504 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:22:52.871091 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59869881-bfb4-4e6d-a524-40676dcbaa62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:22:52.871706 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59869881-bfb4-4e6d-a524-40676dcbaa62 admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:22:52.871986 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59869881-bfb4-4e6d-a524-40676dcbaa62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:22:52.872259 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59869881-bfb4-4e6d-a524-40676dcbaa62 admin admin] PATH_INFO: `/v3/users` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:22:52.872884 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59869881-bfb4-4e6d-a524-40676dcbaa62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:22:52.874827 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59869881-bfb4-4e6d-a524-40676dcbaa62 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:22:52.959679 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1176/2365] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:22:52 2022] POST /identity/v3/users => generated 282 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:22:52.964188 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6283632e-943a-4426-973e-8a88c532c064 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:22:52.974640 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6283632e-943a-4426-973e-8a88c532c064 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:22:52.979455 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fb3ccd-46dc-4c8e-95a6-5b588793589a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:22:52.980051 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fb3ccd-46dc-4c8e-95a6-5b588793589a admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:22:52.980334 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fb3ccd-46dc-4c8e-95a6-5b588793589a admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:22:52.980610 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fb3ccd-46dc-4c8e-95a6-5b588793589a admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:22:52.981252 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fb3ccd-46dc-4c8e-95a6-5b588793589a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:22:52.983030 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fb3ccd-46dc-4c8e-95a6-5b588793589a admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:22:52.993346 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1190/2366] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:22:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:22:52.997696 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59869881-bfb4-4e6d-a524-40676dcbaa62 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:22:53.008247 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59869881-bfb4-4e6d-a524-40676dcbaa62 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:22:53.013599 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8644122f-072e-4243-870f-0b9922698eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:22:53.014213 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8644122f-072e-4243-870f-0b9922698eca admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:22:53.014539 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8644122f-072e-4243-870f-0b9922698eca admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:22:53.014813 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8644122f-072e-4243-870f-0b9922698eca admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:22:53.015456 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8644122f-072e-4243-870f-0b9922698eca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:22:53.017049 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8644122f-072e-4243-870f-0b9922698eca admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:22:53.026850 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1177/2367] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:22:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:22:53.031279 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fb3ccd-46dc-4c8e-95a6-5b588793589a admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:22:53.042236 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fb3ccd-46dc-4c8e-95a6-5b588793589a admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:22:53.046963 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53bec8d-53ee-474b-93e6-da2c3995ffe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:22:53.047627 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53bec8d-53ee-474b-93e6-da2c3995ffe4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:22:53.047909 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53bec8d-53ee-474b-93e6-da2c3995ffe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:22:53.048186 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53bec8d-53ee-474b-93e6-da2c3995ffe4 admin admin] PATH_INFO: `/v3/projects/5105fcc7e7644d47a5bb222eec0ef82b/users/8d90ab33fcae49698918a20c68b42e18/roles/3255f77192524a16bc405173106871fc` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:22:53.063752 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53bec8d-53ee-474b-93e6-da2c3995ffe4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5105fcc7e7644d47a5bb222eec0ef82b, user_id=8d90ab33fcae49698918a20c68b42e18, role_id=3255f77192524a16bc405173106871fc)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:22:53.066452 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53bec8d-53ee-474b-93e6-da2c3995ffe4 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:22:53.150699 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1191/2368] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:22:53 2022] PUT /identity/v3/projects/5105fcc7e7644d47a5bb222eec0ef82b/users/8d90ab33fcae49698918a20c68b42e18/roles/3255f77192524a16bc405173106871fc => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:22:53.162222 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cc2cc2-dd58-4e52-9c4a-d46b451f44e1 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:22:53.162557 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cc2cc2-dd58-4e52-9c4a-d46b451f44e1 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:22:53.162834 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cc2cc2-dd58-4e52-9c4a-d46b451f44e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:22:53.190649 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-29cc2cc2-dd58-4e52-9c4a-d46b451f44e1 None None] MFA Rules not processed for user `8d90ab33fcae49698918a20c68b42e18`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:22:53.191851 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-29cc2cc2-dd58-4e52-9c4a-d46b451f44e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:22:53.255567 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1178/2369] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:22:53 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:22:53.266710 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53bec8d-53ee-474b-93e6-da2c3995ffe4 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:22:53.268426 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-e53bec8d-53ee-474b-93e6-da2c3995ffe4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:22:53.298864 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:22:53.324620 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53bec8d-53ee-474b-93e6-da2c3995ffe4 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:22:53.328338 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cfff4d-ad2c-48e0-9b18-2dfa89bdccff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60193ba9e1c14df3bc9eebcc6f28f2d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:22:53.328948 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cfff4d-ad2c-48e0-9b18-2dfa89bdccff service cinder] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:22:53.329230 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cfff4d-ad2c-48e0-9b18-2dfa89bdccff service cinder] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:22:53.329506 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cfff4d-ad2c-48e0-9b18-2dfa89bdccff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:22:53.337758 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cfff4d-ad2c-48e0-9b18-2dfa89bdccff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:22:53.339211 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cfff4d-ad2c-48e0-9b18-2dfa89bdccff service cinder] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:22:53.347857 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1192/2370] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:22:53 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:22:56.248853 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cc2cc2-dd58-4e52-9c4a-d46b451f44e1 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:22:56.254790 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-29cc2cc2-dd58-4e52-9c4a-d46b451f44e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:22:56.320027 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cc2cc2-dd58-4e52-9c4a-d46b451f44e1 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:22:56.324865 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828296bb-db7d-4125-8ff5-4cdc006cdcfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:22:56.325492 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828296bb-db7d-4125-8ff5-4cdc006cdcfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:22:56.325773 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828296bb-db7d-4125-8ff5-4cdc006cdcfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:22:56.326154 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828296bb-db7d-4125-8ff5-4cdc006cdcfa admin admin] PATH_INFO: `/v3/users/ff639b85df1d4127a7ec3e9c716224c5` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:22:56.327231 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828296bb-db7d-4125-8ff5-4cdc006cdcfa admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff639b85df1d4127a7ec3e9c716224c5)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:22:56.329206 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828296bb-db7d-4125-8ff5-4cdc006cdcfa admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:22:56.344343 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:22:56.496022 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-828296bb-db7d-4125-8ff5-4cdc006cdcfa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff639b85df1d4127a7ec3e9c716224c5'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:22:56.555530 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-828296bb-db7d-4125-8ff5-4cdc006cdcfa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff639b85df1d4127a7ec3e9c716224c5'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:22:56.556199 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-828296bb-db7d-4125-8ff5-4cdc006cdcfa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff639b85df1d4127a7ec3e9c716224c5'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:22:56.556882 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-828296bb-db7d-4125-8ff5-4cdc006cdcfa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff639b85df1d4127a7ec3e9c716224c5'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:22:56.569607 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-828296bb-db7d-4125-8ff5-4cdc006cdcfa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff639b85df1d4127a7ec3e9c716224c5'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:22:56.582714 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1179/2371] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:22:56 2022] DELETE /identity/v3/users/ff639b85df1d4127a7ec3e9c716224c5 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:22:56.587874 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c1feb9-4b53-4a25-bfae-d31917b11a83 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:22:56.591125 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c1feb9-4b53-4a25-bfae-d31917b11a83 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:22:56.591125 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c1feb9-4b53-4a25-bfae-d31917b11a83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:22:56.608440 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-d0c1feb9-4b53-4a25-bfae-d31917b11a83 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:22:56.609636 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-d0c1feb9-4b53-4a25-bfae-d31917b11a83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:22:56.667896 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1193/2372] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:22:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:22:56.678374 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828296bb-db7d-4125-8ff5-4cdc006cdcfa admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:22:56.679794 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-828296bb-db7d-4125-8ff5-4cdc006cdcfa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:22:56.737704 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828296bb-db7d-4125-8ff5-4cdc006cdcfa admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:22:56.741326 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c00140-112e-4fb5-a251-d8810e5bfeba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:22:56.741937 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c00140-112e-4fb5-a251-d8810e5bfeba service neutron] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:22:56.742219 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c00140-112e-4fb5-a251-d8810e5bfeba service neutron] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:22:56.742548 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c00140-112e-4fb5-a251-d8810e5bfeba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:22:56.751771 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c00140-112e-4fb5-a251-d8810e5bfeba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:22:56.753213 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c00140-112e-4fb5-a251-d8810e5bfeba service neutron] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:22:56.763310 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1180/2373] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:22:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:22:57.140183 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c1feb9-4b53-4a25-bfae-d31917b11a83 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:22:57.141765 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-d0c1feb9-4b53-4a25-bfae-d31917b11a83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:22:57.155223 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c1feb9-4b53-4a25-bfae-d31917b11a83 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:22:57.160566 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e609a52c-2512-4d0f-a032-807217c74403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:22:57.161403 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e609a52c-2512-4d0f-a032-807217c74403 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:22:57.161685 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e609a52c-2512-4d0f-a032-807217c74403 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:22:57.162157 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e609a52c-2512-4d0f-a032-807217c74403 admin admin] PATH_INFO: `/v3/projects/dd784a04c8534d4bbd596a1c45a52d31` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:22:57.163440 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e609a52c-2512-4d0f-a032-807217c74403 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd784a04c8534d4bbd596a1c45a52d31)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:22:57.165930 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e609a52c-2512-4d0f-a032-807217c74403 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:22:57.241434 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-e609a52c-2512-4d0f-a032-807217c74403 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd784a04c8534d4bbd596a1c45a52d31', 'request_id': 'req-e609a52c-2512-4d0f-a032-807217c74403'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:22:57.298372 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-e609a52c-2512-4d0f-a032-807217c74403 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd784a04c8534d4bbd596a1c45a52d31', 'request_id': 'req-e609a52c-2512-4d0f-a032-807217c74403'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:22:57.304510 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-e609a52c-2512-4d0f-a032-807217c74403 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd784a04c8534d4bbd596a1c45a52d31', 'request_id': 'req-e609a52c-2512-4d0f-a032-807217c74403'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:22:57.315537 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-e609a52c-2512-4d0f-a032-807217c74403 admin admin] The token cache is being invalidate because project dd784a04c8534d4bbd596a1c45a52d31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:22:57.315830 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-e609a52c-2512-4d0f-a032-807217c74403 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:22:57.316472 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-e609a52c-2512-4d0f-a032-807217c74403 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:22:57.317950 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1194/2374] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:22:57 2022] DELETE /identity/v3/projects/dd784a04c8534d4bbd596a1c45a52d31 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:23:05.431523 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6690b4-ac6a-4a1b-9dfa-c65583ff4051 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:23:05.433107 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6690b4-ac6a-4a1b-9dfa-c65583ff4051 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:23:05.433390 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6690b4-ac6a-4a1b-9dfa-c65583ff4051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:23:05.452818 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-6e6690b4-ac6a-4a1b-9dfa-c65583ff4051 None None] MFA Rules not processed for user `d0a0e9c327564a3db8b89ea706f0fb36`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:23:05.454014 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-6e6690b4-ac6a-4a1b-9dfa-c65583ff4051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:23:05.510995 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1181/2375] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:23:05 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:23:06.916212 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e609a52c-2512-4d0f-a032-807217c74403 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:23:06.917720 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-e609a52c-2512-4d0f-a032-807217c74403 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:23:06.971276 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:23:06.983178 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e609a52c-2512-4d0f-a032-807217c74403 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:23:06.988045 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f10a62f-7609-4dfa-b8ac-0a21194eccd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:23:06.988713 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f10a62f-7609-4dfa-b8ac-0a21194eccd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:23:06.988993 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f10a62f-7609-4dfa-b8ac-0a21194eccd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:23:06.989268 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f10a62f-7609-4dfa-b8ac-0a21194eccd3 admin admin] PATH_INFO: `/v3/users/832a8721625f40759baf0044867664cd` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:23:06.990423 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f10a62f-7609-4dfa-b8ac-0a21194eccd3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=832a8721625f40759baf0044867664cd)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:23:06.992351 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f10a62f-7609-4dfa-b8ac-0a21194eccd3 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:23:07.147263 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-8f10a62f-7609-4dfa-b8ac-0a21194eccd3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '832a8721625f40759baf0044867664cd'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:23:07.200788 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-8f10a62f-7609-4dfa-b8ac-0a21194eccd3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '832a8721625f40759baf0044867664cd'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:23:07.201490 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-8f10a62f-7609-4dfa-b8ac-0a21194eccd3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '832a8721625f40759baf0044867664cd'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:23:07.208819 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-8f10a62f-7609-4dfa-b8ac-0a21194eccd3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '832a8721625f40759baf0044867664cd'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:23:07.220964 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-8f10a62f-7609-4dfa-b8ac-0a21194eccd3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '832a8721625f40759baf0044867664cd'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:23:07.227729 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1195/2376] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:23:06 2022] DELETE /identity/v3/users/832a8721625f40759baf0044867664cd => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:23:07.232148 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d132b2-df31-45d1-95e5-9730b617781b None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:23:07.232910 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d132b2-df31-45d1-95e5-9730b617781b None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:23:07.233029 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d132b2-df31-45d1-95e5-9730b617781b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:23:07.252357 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-c2d132b2-df31-45d1-95e5-9730b617781b None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:23:07.253607 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-c2d132b2-df31-45d1-95e5-9730b617781b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:23:07.291702 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:23:07.312067 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1182/2377] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:23:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:23:07.321919 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f10a62f-7609-4dfa-b8ac-0a21194eccd3 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:23:07.323368 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-8f10a62f-7609-4dfa-b8ac-0a21194eccd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:23:07.383228 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f10a62f-7609-4dfa-b8ac-0a21194eccd3 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:23:07.387080 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baad850-06a1-4cab-8c15-9c41ca278f58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:23:07.387688 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baad850-06a1-4cab-8c15-9c41ca278f58 service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:23:07.388005 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baad850-06a1-4cab-8c15-9c41ca278f58 service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:23:07.388245 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baad850-06a1-4cab-8c15-9c41ca278f58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:23:07.397658 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baad850-06a1-4cab-8c15-9c41ca278f58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:23:07.399135 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baad850-06a1-4cab-8c15-9c41ca278f58 service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:23:07.408749 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1196/2378] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:23:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:23:07.834533 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d132b2-df31-45d1-95e5-9730b617781b None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:23:07.836003 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-c2d132b2-df31-45d1-95e5-9730b617781b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:23:07.857715 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d132b2-df31-45d1-95e5-9730b617781b None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:23:07.866299 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cff34b9-6d4d-4648-99b6-ae9e50eaa17d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:23:07.867126 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cff34b9-6d4d-4648-99b6-ae9e50eaa17d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:23:07.867736 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cff34b9-6d4d-4648-99b6-ae9e50eaa17d admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:23:07.868064 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cff34b9-6d4d-4648-99b6-ae9e50eaa17d admin admin] PATH_INFO: `/v3/projects/2ccf846bb93a4e7d87054ec280216c84` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:23:07.869572 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cff34b9-6d4d-4648-99b6-ae9e50eaa17d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ccf846bb93a4e7d87054ec280216c84)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:23:07.873020 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cff34b9-6d4d-4648-99b6-ae9e50eaa17d admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:23:07.991956 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-8cff34b9-6d4d-4648-99b6-ae9e50eaa17d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ccf846bb93a4e7d87054ec280216c84', 'request_id': 'req-8cff34b9-6d4d-4648-99b6-ae9e50eaa17d'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:23:08.064255 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-8cff34b9-6d4d-4648-99b6-ae9e50eaa17d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ccf846bb93a4e7d87054ec280216c84', 'request_id': 'req-8cff34b9-6d4d-4648-99b6-ae9e50eaa17d'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:23:08.071953 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-8cff34b9-6d4d-4648-99b6-ae9e50eaa17d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ccf846bb93a4e7d87054ec280216c84', 'request_id': 'req-8cff34b9-6d4d-4648-99b6-ae9e50eaa17d'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:23:08.083891 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-8cff34b9-6d4d-4648-99b6-ae9e50eaa17d admin admin] The token cache is being invalidate because project 2ccf846bb93a4e7d87054ec280216c84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81995) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:23:08.084184 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-8cff34b9-6d4d-4648-99b6-ae9e50eaa17d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:23:08.084949 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-8cff34b9-6d4d-4648-99b6-ae9e50eaa17d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:23:08.087234 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1183/2379] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:23:07 2022] DELETE /identity/v3/projects/2ccf846bb93a4e7d87054ec280216c84 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:23:17.404328 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=55,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:23:18.080905 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=35,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:23:31.341475 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baad850-06a1-4cab-8c15-9c41ca278f58 service neutron] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:23:31.343590 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-9baad850-06a1-4cab-8c15-9c41ca278f58 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:23:31.354928 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baad850-06a1-4cab-8c15-9c41ca278f58 service neutron] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:23:31.408455 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740ae8e2-4789-4f1e-8f03-23c3dfbd93bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:23:31.409068 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740ae8e2-4789-4f1e-8f03-23c3dfbd93bd service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:23:31.409351 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740ae8e2-4789-4f1e-8f03-23c3dfbd93bd service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:23:31.409627 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740ae8e2-4789-4f1e-8f03-23c3dfbd93bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:23:31.411419 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-740ae8e2-4789-4f1e-8f03-23c3dfbd93bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:23:31.482139 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740ae8e2-4789-4f1e-8f03-23c3dfbd93bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:23:31.483643 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740ae8e2-4789-4f1e-8f03-23c3dfbd93bd service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:23:31.493067 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1197/2380] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:23:31 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:23:41.488870 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=50 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:23:49.033221 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cff34b9-6d4d-4648-99b6-ae9e50eaa17d admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:23:49.034736 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-8cff34b9-6d4d-4648-99b6-ae9e50eaa17d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:23:49.047150 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cff34b9-6d4d-4648-99b6-ae9e50eaa17d admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:23:49.050884 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bdea36-b3c7-4af9-b3d6-a0aa4a4cfacb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:23:49.051495 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bdea36-b3c7-4af9-b3d6-a0aa4a4cfacb service neutron] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:23:49.051773 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bdea36-b3c7-4af9-b3d6-a0aa4a4cfacb service neutron] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:23:49.052044 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bdea36-b3c7-4af9-b3d6-a0aa4a4cfacb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:23:49.053737 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-31bdea36-b3c7-4af9-b3d6-a0aa4a4cfacb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:23:49.125453 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bdea36-b3c7-4af9-b3d6-a0aa4a4cfacb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:23:49.126914 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bdea36-b3c7-4af9-b3d6-a0aa4a4cfacb service neutron] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:23:49.137753 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1184/2381] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:23:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:23:50.317209 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740ae8e2-4789-4f1e-8f03-23c3dfbd93bd service neutron] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:23:50.326324 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740ae8e2-4789-4f1e-8f03-23c3dfbd93bd service neutron] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:23:50.330731 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dc4ab7-2799-4680-99d0-ec27aa4ad57d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:23:50.331332 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dc4ab7-2799-4680-99d0-ec27aa4ad57d service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:23:50.331611 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dc4ab7-2799-4680-99d0-ec27aa4ad57d service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:23:50.331941 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dc4ab7-2799-4680-99d0-ec27aa4ad57d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:23:50.333714 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-f3dc4ab7-2799-4680-99d0-ec27aa4ad57d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:23:50.347408 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dc4ab7-2799-4680-99d0-ec27aa4ad57d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:23:50.348871 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dc4ab7-2799-4680-99d0-ec27aa4ad57d service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:23:50.359702 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1198/2382] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:23:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:23:50.532059 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bdea36-b3c7-4af9-b3d6-a0aa4a4cfacb service neutron] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:23:50.533449 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-31bdea36-b3c7-4af9-b3d6-a0aa4a4cfacb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:23:50.715945 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bdea36-b3c7-4af9-b3d6-a0aa4a4cfacb service neutron] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:23:50.720444 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4478e641-e4cf-4819-bae2-17cf3a1dcfcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:23:50.721074 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4478e641-e4cf-4819-bae2-17cf3a1dcfcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:23:50.721355 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4478e641-e4cf-4819-bae2-17cf3a1dcfcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:23:50.721627 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4478e641-e4cf-4819-bae2-17cf3a1dcfcc admin admin] PATH_INFO: `/v3/users/8d90ab33fcae49698918a20c68b42e18` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:23:50.722763 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4478e641-e4cf-4819-bae2-17cf3a1dcfcc admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d90ab33fcae49698918a20c68b42e18)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:23:50.724672 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4478e641-e4cf-4819-bae2-17cf3a1dcfcc admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:23:50.910488 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-4478e641-e4cf-4819-bae2-17cf3a1dcfcc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d90ab33fcae49698918a20c68b42e18'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:23:50.975632 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-4478e641-e4cf-4819-bae2-17cf3a1dcfcc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d90ab33fcae49698918a20c68b42e18'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:23:50.976200 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-4478e641-e4cf-4819-bae2-17cf3a1dcfcc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d90ab33fcae49698918a20c68b42e18'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:23:50.976776 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-4478e641-e4cf-4819-bae2-17cf3a1dcfcc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d90ab33fcae49698918a20c68b42e18'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:23:50.989336 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-4478e641-e4cf-4819-bae2-17cf3a1dcfcc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d90ab33fcae49698918a20c68b42e18'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:23:51.001187 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1185/2383] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:23:50 2022] DELETE /identity/v3/users/8d90ab33fcae49698918a20c68b42e18 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:23:51.005724 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb65923d-6018-4b61-9cb9-7c5df3ea342b None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:23:51.006014 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb65923d-6018-4b61-9cb9-7c5df3ea342b None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:23:51.006323 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb65923d-6018-4b61-9cb9-7c5df3ea342b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:23:51.025461 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-eb65923d-6018-4b61-9cb9-7c5df3ea342b None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:23:51.026721 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-eb65923d-6018-4b61-9cb9-7c5df3ea342b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:23:51.085737 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1199/2384] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:23:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:23:51.095832 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4478e641-e4cf-4819-bae2-17cf3a1dcfcc admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:23:51.097221 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-4478e641-e4cf-4819-bae2-17cf3a1dcfcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:23:51.103676 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:23:51.153907 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4478e641-e4cf-4819-bae2-17cf3a1dcfcc admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:23:51.157988 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bc4b72-6e43-4566-b490-9fad6c6a50ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:23:51.158849 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bc4b72-6e43-4566-b490-9fad6c6a50ee service neutron] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:23:51.159249 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bc4b72-6e43-4566-b490-9fad6c6a50ee service neutron] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:23:51.159887 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bc4b72-6e43-4566-b490-9fad6c6a50ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:23:51.168890 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bc4b72-6e43-4566-b490-9fad6c6a50ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:23:51.171305 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bc4b72-6e43-4566-b490-9fad6c6a50ee service neutron] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:23:51.180810 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1186/2385] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:23:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:23:51.598008 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb65923d-6018-4b61-9cb9-7c5df3ea342b None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:23:51.598008 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-eb65923d-6018-4b61-9cb9-7c5df3ea342b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:23:51.598757 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb65923d-6018-4b61-9cb9-7c5df3ea342b None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:23:51.609409 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72534c51-a70e-4d55-87fe-c6d9b511a51e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:23:51.610159 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72534c51-a70e-4d55-87fe-c6d9b511a51e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:23:51.610528 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72534c51-a70e-4d55-87fe-c6d9b511a51e admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:23:51.610866 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72534c51-a70e-4d55-87fe-c6d9b511a51e admin admin] PATH_INFO: `/v3/projects/5105fcc7e7644d47a5bb222eec0ef82b` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:23:51.612313 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72534c51-a70e-4d55-87fe-c6d9b511a51e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5105fcc7e7644d47a5bb222eec0ef82b)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:23:51.614691 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72534c51-a70e-4d55-87fe-c6d9b511a51e admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:23:51.724630 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-72534c51-a70e-4d55-87fe-c6d9b511a51e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5105fcc7e7644d47a5bb222eec0ef82b', 'request_id': 'req-72534c51-a70e-4d55-87fe-c6d9b511a51e'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:23:51.804597 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-72534c51-a70e-4d55-87fe-c6d9b511a51e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5105fcc7e7644d47a5bb222eec0ef82b', 'request_id': 'req-72534c51-a70e-4d55-87fe-c6d9b511a51e'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:23:51.810586 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-72534c51-a70e-4d55-87fe-c6d9b511a51e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5105fcc7e7644d47a5bb222eec0ef82b', 'request_id': 'req-72534c51-a70e-4d55-87fe-c6d9b511a51e'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:23:51.822354 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-72534c51-a70e-4d55-87fe-c6d9b511a51e admin admin] The token cache is being invalidate because project 5105fcc7e7644d47a5bb222eec0ef82b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:23:51.822694 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-72534c51-a70e-4d55-87fe-c6d9b511a51e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:23:51.823271 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-72534c51-a70e-4d55-87fe-c6d9b511a51e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:23:51.824753 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1200/2386] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:23:51 2022] DELETE /identity/v3/projects/5105fcc7e7644d47a5bb222eec0ef82b => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:23:51.830509 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d2776c-0599-44f8-83e4-c6591122b0a5 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:23:51.830995 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d2776c-0599-44f8-83e4-c6591122b0a5 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:23:51.831133 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d2776c-0599-44f8-83e4-c6591122b0a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:23:51.849849 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-10d2776c-0599-44f8-83e4-c6591122b0a5 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:23:51.851068 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-10d2776c-0599-44f8-83e4-c6591122b0a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:23:51.917254 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1187/2387] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:23:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:23:51.927791 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25615d7-6747-4351-bc43-52e3ef294719 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:23:51.928076 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25615d7-6747-4351-bc43-52e3ef294719 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:23:51.928352 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25615d7-6747-4351-bc43-52e3ef294719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:23:51.948618 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-a25615d7-6747-4351-bc43-52e3ef294719 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:23:51.949831 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-a25615d7-6747-4351-bc43-52e3ef294719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:23:51.958144 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1201/2388] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:23:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:23:51.962378 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d2776c-0599-44f8-83e4-c6591122b0a5 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:23:51.971983 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d2776c-0599-44f8-83e4-c6591122b0a5 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:23:51.976486 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3485f910-d82d-44cc-b068-9a004ad7ea9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:23:51.977091 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3485f910-d82d-44cc-b068-9a004ad7ea9a admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:23:51.977361 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3485f910-d82d-44cc-b068-9a004ad7ea9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:23:51.977632 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3485f910-d82d-44cc-b068-9a004ad7ea9a admin admin] PATH_INFO: `/v3/domains` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:23:51.978270 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3485f910-d82d-44cc-b068-9a004ad7ea9a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:23:51.978602 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3485f910-d82d-44cc-b068-9a004ad7ea9a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:23:51.980193 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3485f910-d82d-44cc-b068-9a004ad7ea9a admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:23:51.991307 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1188/2389] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Aug 25 03:23:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:23:51.995524 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25615d7-6747-4351-bc43-52e3ef294719 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:23:52.004929 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25615d7-6747-4351-bc43-52e3ef294719 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:23:52.009442 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cafca39-80e4-46f3-8dbc-92c56312b41e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:23:52.010028 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cafca39-80e4-46f3-8dbc-92c56312b41e admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:23:52.010328 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cafca39-80e4-46f3-8dbc-92c56312b41e admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:23:52.010601 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cafca39-80e4-46f3-8dbc-92c56312b41e admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:23:52.011231 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cafca39-80e4-46f3-8dbc-92c56312b41e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:23:52.013101 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cafca39-80e4-46f3-8dbc-92c56312b41e admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:23:52.080114 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1202/2390] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:23:51 2022] POST /identity/v3/projects => generated 385 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:23:52.084199 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3485f910-d82d-44cc-b068-9a004ad7ea9a admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:23:52.094023 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3485f910-d82d-44cc-b068-9a004ad7ea9a admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:23:52.149745 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadca578-9b2d-45a4-9c36-a1835a25824c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:23:52.150393 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadca578-9b2d-45a4-9c36-a1835a25824c admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:23:52.150674 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadca578-9b2d-45a4-9c36-a1835a25824c admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:23:52.150949 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadca578-9b2d-45a4-9c36-a1835a25824c admin admin] PATH_INFO: `/v3/users` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:23:52.151608 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadca578-9b2d-45a4-9c36-a1835a25824c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:23:52.153515 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadca578-9b2d-45a4-9c36-a1835a25824c admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:23:52.249754 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1189/2391] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:23:52 2022] POST /identity/v3/users => generated 293 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:23:52.253917 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cafca39-80e4-46f3-8dbc-92c56312b41e admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:23:52.264372 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cafca39-80e4-46f3-8dbc-92c56312b41e admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:23:52.269163 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9836846-7c0c-43cd-8b44-9f54c516b44c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:23:52.269784 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9836846-7c0c-43cd-8b44-9f54c516b44c admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:23:52.270066 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9836846-7c0c-43cd-8b44-9f54c516b44c admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:23:52.270382 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9836846-7c0c-43cd-8b44-9f54c516b44c admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:23:52.271036 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9836846-7c0c-43cd-8b44-9f54c516b44c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:23:52.272634 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9836846-7c0c-43cd-8b44-9f54c516b44c admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:23:52.282825 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1203/2392] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:23:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:23:52.287082 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadca578-9b2d-45a4-9c36-a1835a25824c admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:23:52.297609 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadca578-9b2d-45a4-9c36-a1835a25824c admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:23:52.302415 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167f3198-4046-4cb8-8482-c5ac2b591716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:23:52.303010 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167f3198-4046-4cb8-8482-c5ac2b591716 admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:23:52.303292 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167f3198-4046-4cb8-8482-c5ac2b591716 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:23:52.303569 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167f3198-4046-4cb8-8482-c5ac2b591716 admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:23:52.304211 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167f3198-4046-4cb8-8482-c5ac2b591716 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:23:52.305797 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167f3198-4046-4cb8-8482-c5ac2b591716 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:23:52.316225 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1190/2393] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:23:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:23:52.320545 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9836846-7c0c-43cd-8b44-9f54c516b44c admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:23:52.336027 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9836846-7c0c-43cd-8b44-9f54c516b44c admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:23:52.336027 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8990abe3-60da-44c3-a5ac-b0007f0572e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:23:52.336027 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8990abe3-60da-44c3-a5ac-b0007f0572e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:23:52.336394 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8990abe3-60da-44c3-a5ac-b0007f0572e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:23:52.336394 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8990abe3-60da-44c3-a5ac-b0007f0572e5 admin admin] PATH_INFO: `/v3/projects/3e91375fe4114bf98447ccfcb6f8475f/users/dd2985023e274f19a868da37aa791859/roles/3255f77192524a16bc405173106871fc` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:23:52.351131 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8990abe3-60da-44c3-a5ac-b0007f0572e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e91375fe4114bf98447ccfcb6f8475f, user_id=dd2985023e274f19a868da37aa791859, role_id=3255f77192524a16bc405173106871fc)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:23:52.353850 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8990abe3-60da-44c3-a5ac-b0007f0572e5 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:23:52.410023 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1204/2394] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:23:52 2022] PUT /identity/v3/projects/3e91375fe4114bf98447ccfcb6f8475f/users/dd2985023e274f19a868da37aa791859/roles/3255f77192524a16bc405173106871fc => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:23:52.414975 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9196048-c610-4362-879a-7f23b1d34446 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:23:52.415264 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9196048-c610-4362-879a-7f23b1d34446 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:23:52.415539 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9196048-c610-4362-879a-7f23b1d34446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:23:52.434000 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-f9196048-c610-4362-879a-7f23b1d34446 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:23:52.435234 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-f9196048-c610-4362-879a-7f23b1d34446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:23:52.443995 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:23:52.492740 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1191/2395] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:23:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:23:52.502853 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8990abe3-60da-44c3-a5ac-b0007f0572e5 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:23:52.504581 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-8990abe3-60da-44c3-a5ac-b0007f0572e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:23:52.514178 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:23:52.561901 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8990abe3-60da-44c3-a5ac-b0007f0572e5 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:23:52.565315 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9766f204-899d-4bd8-966b-627949fcbc92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:23:52.565919 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9766f204-899d-4bd8-966b-627949fcbc92 service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:23:52.566199 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9766f204-899d-4bd8-966b-627949fcbc92 service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:23:52.566502 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9766f204-899d-4bd8-966b-627949fcbc92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:23:52.575977 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9766f204-899d-4bd8-966b-627949fcbc92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:23:52.578175 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9766f204-899d-4bd8-966b-627949fcbc92 service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:23:52.588235 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1205/2396] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:23:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 25 03:23:57.739449 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe712657-48e3-467a-9bb7-ff8ccda64576 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:23:57.739449 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe712657-48e3-467a-9bb7-ff8ccda64576 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:23:57.739449 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe712657-48e3-467a-9bb7-ff8ccda64576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:23:57.774621 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-fe712657-48e3-467a-9bb7-ff8ccda64576 None None] MFA Rules not processed for user `dd2985023e274f19a868da37aa791859`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:23:57.774621 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-fe712657-48e3-467a-9bb7-ff8ccda64576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:23:57.891058 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1192/2397] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:23:57 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:23:57.895445 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9766f204-899d-4bd8-966b-627949fcbc92 service neutron] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:23:57.905953 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9766f204-899d-4bd8-966b-627949fcbc92 service neutron] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:23:57.913725 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e534a7a8-d69e-4f26-b6fc-8ff4a19cc24a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:23:57.914393 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e534a7a8-d69e-4f26-b6fc-8ff4a19cc24a admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:23:57.914681 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e534a7a8-d69e-4f26-b6fc-8ff4a19cc24a admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:23:57.914957 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e534a7a8-d69e-4f26-b6fc-8ff4a19cc24a admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:23:57.915611 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e534a7a8-d69e-4f26-b6fc-8ff4a19cc24a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:23:57.917559 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e534a7a8-d69e-4f26-b6fc-8ff4a19cc24a admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:23:58.031652 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1206/2398] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:23:57 2022] POST /identity/v3/projects => generated 383 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:23:58.033714 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe712657-48e3-467a-9bb7-ff8ccda64576 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:23:58.045839 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe712657-48e3-467a-9bb7-ff8ccda64576 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:23:58.105353 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f748cd-c547-4a9d-b198-55b056e8693f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:23:58.105972 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f748cd-c547-4a9d-b198-55b056e8693f admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:23:58.106250 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f748cd-c547-4a9d-b198-55b056e8693f admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:23:58.106560 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f748cd-c547-4a9d-b198-55b056e8693f admin admin] PATH_INFO: `/v3/users` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:23:58.107196 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f748cd-c547-4a9d-b198-55b056e8693f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:23:58.109650 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f748cd-c547-4a9d-b198-55b056e8693f admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:23:58.206920 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1193/2399] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:23:58 2022] POST /identity/v3/users => generated 298 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:23:58.208562 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e534a7a8-d69e-4f26-b6fc-8ff4a19cc24a admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:23:58.222622 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e534a7a8-d69e-4f26-b6fc-8ff4a19cc24a admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:23:58.227399 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa09f7f5-d566-45f6-a38b-d4eb24464aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:23:58.228187 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa09f7f5-d566-45f6-a38b-d4eb24464aec admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:23:58.228446 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa09f7f5-d566-45f6-a38b-d4eb24464aec admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:23:58.228737 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa09f7f5-d566-45f6-a38b-d4eb24464aec admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:23:58.229400 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa09f7f5-d566-45f6-a38b-d4eb24464aec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:23:58.233242 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa09f7f5-d566-45f6-a38b-d4eb24464aec admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:23:58.244857 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1207/2400] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:23:58 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:23:58.249612 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f748cd-c547-4a9d-b198-55b056e8693f admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:23:58.259501 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f748cd-c547-4a9d-b198-55b056e8693f admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:23:58.264152 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63161225-8db4-4570-9d0e-0b136b66edfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:23:58.264797 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63161225-8db4-4570-9d0e-0b136b66edfb admin admin] REQUEST_METHOD: `PUT` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:23:58.265078 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63161225-8db4-4570-9d0e-0b136b66edfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:23:58.265352 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63161225-8db4-4570-9d0e-0b136b66edfb admin admin] PATH_INFO: `/v3/projects/d4cbf559385341bd9cd2fa5388a2c7bb/users/f7f83c84f8814829aa7114ad74c50232/roles/62f19ea272b447ccaab1576cd9bf4fd9` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:23:58.283009 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63161225-8db4-4570-9d0e-0b136b66edfb admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4cbf559385341bd9cd2fa5388a2c7bb, user_id=f7f83c84f8814829aa7114ad74c50232, role_id=62f19ea272b447ccaab1576cd9bf4fd9)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:23:58.285658 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63161225-8db4-4570-9d0e-0b136b66edfb admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:23:58.371214 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1194/2401] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:23:58 2022] PUT /identity/v3/projects/d4cbf559385341bd9cd2fa5388a2c7bb/users/f7f83c84f8814829aa7114ad74c50232/roles/62f19ea272b447ccaab1576cd9bf4fd9 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:23:58.375956 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa09f7f5-d566-45f6-a38b-d4eb24464aec admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:23:58.386647 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa09f7f5-d566-45f6-a38b-d4eb24464aec admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:23:58.451774 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9487e3f-16ed-453a-ba55-d354b0331668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:23:58.452392 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9487e3f-16ed-453a-ba55-d354b0331668 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:23:58.452667 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9487e3f-16ed-453a-ba55-d354b0331668 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:23:58.452935 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9487e3f-16ed-453a-ba55-d354b0331668 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:23:58.453686 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9487e3f-16ed-453a-ba55-d354b0331668 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:23:58.455415 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9487e3f-16ed-453a-ba55-d354b0331668 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:23:58.466069 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1208/2402] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:23:58 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:23:58.471014 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63161225-8db4-4570-9d0e-0b136b66edfb admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:23:58.481616 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63161225-8db4-4570-9d0e-0b136b66edfb admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:23:58.487215 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377b8190-e676-48f3-9d99-2e07d74662af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:23:58.487811 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377b8190-e676-48f3-9d99-2e07d74662af admin admin] REQUEST_METHOD: `PUT` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:23:58.488089 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377b8190-e676-48f3-9d99-2e07d74662af admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:23:58.488361 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377b8190-e676-48f3-9d99-2e07d74662af admin admin] PATH_INFO: `/v3/projects/d4cbf559385341bd9cd2fa5388a2c7bb/users/f7f83c84f8814829aa7114ad74c50232/roles/62f19ea272b447ccaab1576cd9bf4fd9` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:23:58.490255 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377b8190-e676-48f3-9d99-2e07d74662af admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4cbf559385341bd9cd2fa5388a2c7bb, user_id=f7f83c84f8814829aa7114ad74c50232, role_id=62f19ea272b447ccaab1576cd9bf4fd9)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:23:58.492999 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377b8190-e676-48f3-9d99-2e07d74662af admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:23:58.496438 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:23:58.510246 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1195/2403] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:23:58 2022] PUT /identity/v3/projects/d4cbf559385341bd9cd2fa5388a2c7bb/users/f7f83c84f8814829aa7114ad74c50232/roles/62f19ea272b447ccaab1576cd9bf4fd9 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:24:03.469367 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9487e3f-16ed-453a-ba55-d354b0331668 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:24:03.472561 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-c9487e3f-16ed-453a-ba55-d354b0331668 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:24:03.556039 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9487e3f-16ed-453a-ba55-d354b0331668 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:24:03.566340 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:24:03.643306 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788f3801-1458-426c-bed6-fcd4a64fe9e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cd171bf4df94f8fab397e42835feeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:24:03.643729 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788f3801-1458-426c-bed6-fcd4a64fe9e0 service nova] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:24:03.643956 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788f3801-1458-426c-bed6-fcd4a64fe9e0 service nova] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:24:03.644832 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788f3801-1458-426c-bed6-fcd4a64fe9e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:24:03.646549 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-788f3801-1458-426c-bed6-fcd4a64fe9e0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:24:03.713087 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788f3801-1458-426c-bed6-fcd4a64fe9e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:24:03.714661 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788f3801-1458-426c-bed6-fcd4a64fe9e0 service nova] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:24:03.725728 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1209/2404] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:24:03 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:24:04.086068 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765bb885-bbb7-44a2-b889-41ec2814235b None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:24:04.086587 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765bb885-bbb7-44a2-b889-41ec2814235b None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:24:04.086880 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765bb885-bbb7-44a2-b889-41ec2814235b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:24:04.117058 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-765bb885-bbb7-44a2-b889-41ec2814235b None None] MFA Rules not processed for user `f7f83c84f8814829aa7114ad74c50232`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:24:04.118535 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-765bb885-bbb7-44a2-b889-41ec2814235b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:24:04.190981 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1196/2405] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:24:04 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:24:04.203485 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788f3801-1458-426c-bed6-fcd4a64fe9e0 service nova] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:24:04.205580 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-788f3801-1458-426c-bed6-fcd4a64fe9e0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:24:04.274981 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788f3801-1458-426c-bed6-fcd4a64fe9e0 service nova] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:24:04.279350 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1035fc-92e2-46b0-b7e9-c09facdd64e2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f89089b390c4b6599e54000c0fa082a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:24:04.280561 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1035fc-92e2-46b0-b7e9-c09facdd64e2 service glance] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:24:04.280876 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1035fc-92e2-46b0-b7e9-c09facdd64e2 service glance] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:24:04.281102 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1035fc-92e2-46b0-b7e9-c09facdd64e2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:24:04.340424 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1035fc-92e2-46b0-b7e9-c09facdd64e2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:24:04.341911 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1035fc-92e2-46b0-b7e9-c09facdd64e2 service glance] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:24:04.352023 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1210/2406] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:24:04 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:24:04.652679 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac73cf5-669d-4f52-b695-de4db40c52c8 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:24:04.652963 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac73cf5-669d-4f52-b695-de4db40c52c8 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:24:04.653240 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac73cf5-669d-4f52-b695-de4db40c52c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:24:04.674780 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-5ac73cf5-669d-4f52-b695-de4db40c52c8 None None] MFA Rules not processed for user `712c35b765b64919aa3eaaf02892590d`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:24:04.681079 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-5ac73cf5-669d-4f52-b695-de4db40c52c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:24:04.733981 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1197/2407] 10.222.0.58 () {58 vars in 1009 bytes} [Thu Aug 25 03:24:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:24:04.888940 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1035fc-92e2-46b0-b7e9-c09facdd64e2 service glance] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:24:04.890494 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-6d1035fc-92e2-46b0-b7e9-c09facdd64e2 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:24:04.895948 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:24:04.976214 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1035fc-92e2-46b0-b7e9-c09facdd64e2 service glance] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:24:04.983848 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c828742-0dd1-4945-8b63-7bae6edbb75d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5047e1a7cd2d416d90b2d8c5a6596522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:24:04.986888 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c828742-0dd1-4945-8b63-7bae6edbb75d service swift] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:24:04.986888 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c828742-0dd1-4945-8b63-7bae6edbb75d service swift] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:24:04.986888 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c828742-0dd1-4945-8b63-7bae6edbb75d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:24:04.994414 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c828742-0dd1-4945-8b63-7bae6edbb75d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:24:04.996403 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c828742-0dd1-4945-8b63-7bae6edbb75d service swift] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:24:05.006231 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1211/2408] 10.222.0.58 () {60 vars in 1384 bytes} [Thu Aug 25 03:24:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 25 03:24:14.727221 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=61 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:24:15.002610 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=31 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:24:15.684257 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd35f48f-ab1b-4a1e-8b6f-4ec5e1e78725 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:24:15.684541 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd35f48f-ab1b-4a1e-8b6f-4ec5e1e78725 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:24:15.684818 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd35f48f-ab1b-4a1e-8b6f-4ec5e1e78725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:24:15.703935 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-bd35f48f-ab1b-4a1e-8b6f-4ec5e1e78725 None None] MFA Rules not processed for user `dd2985023e274f19a868da37aa791859`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:24:15.705271 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-bd35f48f-ab1b-4a1e-8b6f-4ec5e1e78725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:24:15.712231 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1198/2409] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:24:15 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:24:15.966271 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c828742-0dd1-4945-8b63-7bae6edbb75d service swift] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:24:15.967732 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-0c828742-0dd1-4945-8b63-7bae6edbb75d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:24:16.025418 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c828742-0dd1-4945-8b63-7bae6edbb75d service swift] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:24:16.028996 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e1be4c-d350-43dd-af5e-ca7ce029b58f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:24:16.029590 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e1be4c-d350-43dd-af5e-ca7ce029b58f service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:24:16.029865 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e1be4c-d350-43dd-af5e-ca7ce029b58f service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:24:16.030134 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e1be4c-d350-43dd-af5e-ca7ce029b58f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:24:16.032010 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-87e1be4c-d350-43dd-af5e-ca7ce029b58f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:24:16.041252 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e1be4c-d350-43dd-af5e-ca7ce029b58f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:24:16.042775 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e1be4c-d350-43dd-af5e-ca7ce029b58f service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:24:16.051404 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1212/2410] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:24:15 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:24:16.445797 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd35f48f-ab1b-4a1e-8b6f-4ec5e1e78725 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:24:16.447281 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-bd35f48f-ab1b-4a1e-8b6f-4ec5e1e78725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:24:16.517145 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd35f48f-ab1b-4a1e-8b6f-4ec5e1e78725 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:24:16.522145 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9156a41e-a16c-402b-9b82-ad6ae0931f68 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b692edd3e8c41289cc42e1afb4a6dd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:24:16.522783 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9156a41e-a16c-402b-9b82-ad6ae0931f68 service placement] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:24:16.523067 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9156a41e-a16c-402b-9b82-ad6ae0931f68 service placement] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:24:16.523338 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9156a41e-a16c-402b-9b82-ad6ae0931f68 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:24:16.525072 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-9156a41e-a16c-402b-9b82-ad6ae0931f68 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:24:16.537141 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9156a41e-a16c-402b-9b82-ad6ae0931f68 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:24:16.538742 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9156a41e-a16c-402b-9b82-ad6ae0931f68 service placement] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:24:16.549126 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1199/2411] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:24:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:24:19.175864 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2fbc79-b517-40a0-9738-30ae0340427b None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:24:19.176333 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2fbc79-b517-40a0-9738-30ae0340427b None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:24:19.176472 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2fbc79-b517-40a0-9738-30ae0340427b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:24:19.199951 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-5a2fbc79-b517-40a0-9738-30ae0340427b None None] MFA Rules not processed for user `712c35b765b64919aa3eaaf02892590d`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:24:19.201314 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-5a2fbc79-b517-40a0-9738-30ae0340427b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:24:19.210006 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1213/2412] 10.222.0.58 () {58 vars in 1009 bytes} [Thu Aug 25 03:24:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:24:19.225987 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9156a41e-a16c-402b-9b82-ad6ae0931f68 service placement] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:24:19.240662 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9156a41e-a16c-402b-9b82-ad6ae0931f68 service placement] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:24:19.246100 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b47882e-22bd-4379-bf0f-af8aadaeb003 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5047e1a7cd2d416d90b2d8c5a6596522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:24:19.246739 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b47882e-22bd-4379-bf0f-af8aadaeb003 service swift] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:24:19.247019 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b47882e-22bd-4379-bf0f-af8aadaeb003 service swift] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:24:19.247310 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b47882e-22bd-4379-bf0f-af8aadaeb003 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:24:19.256601 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b47882e-22bd-4379-bf0f-af8aadaeb003 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:24:19.258206 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b47882e-22bd-4379-bf0f-af8aadaeb003 service swift] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:24:19.266429 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1200/2413] 10.222.0.58 () {60 vars in 1384 bytes} [Thu Aug 25 03:24:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 25 03:24:29.190825 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=29 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:24:29.263132 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=41 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:24:33.401962 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2fbc79-b517-40a0-9738-30ae0340427b None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:24:33.412072 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2fbc79-b517-40a0-9738-30ae0340427b None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:24:33.415690 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b6d9d7-33fd-4d60-8483-d00cb06e6d58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:24:33.416298 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b6d9d7-33fd-4d60-8483-d00cb06e6d58 service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:24:33.416581 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b6d9d7-33fd-4d60-8483-d00cb06e6d58 service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:24:33.416861 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b6d9d7-33fd-4d60-8483-d00cb06e6d58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:24:33.420472 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-d0b6d9d7-33fd-4d60-8483-d00cb06e6d58 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:24:33.429438 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b6d9d7-33fd-4d60-8483-d00cb06e6d58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:24:33.431025 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b6d9d7-33fd-4d60-8483-d00cb06e6d58 service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:24:33.439090 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1214/2414] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:24:33 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:24:34.368412 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b47882e-22bd-4379-bf0f-af8aadaeb003 service swift] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:24:34.384281 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b47882e-22bd-4379-bf0f-af8aadaeb003 service swift] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:24:34.389830 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee586b9-679b-4ea4-b5d7-daa3cc2cb1c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cd171bf4df94f8fab397e42835feeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:24:34.390470 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee586b9-679b-4ea4-b5d7-daa3cc2cb1c8 service nova] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:24:34.390750 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee586b9-679b-4ea4-b5d7-daa3cc2cb1c8 service nova] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:24:34.391099 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee586b9-679b-4ea4-b5d7-daa3cc2cb1c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:24:34.392846 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-0ee586b9-679b-4ea4-b5d7-daa3cc2cb1c8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:24:34.405693 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee586b9-679b-4ea4-b5d7-daa3cc2cb1c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:24:34.407432 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee586b9-679b-4ea4-b5d7-daa3cc2cb1c8 service nova] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:24:34.418404 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1201/2415] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:24:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:24:37.030380 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b6d9d7-33fd-4d60-8483-d00cb06e6d58 service neutron] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:24:37.031978 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-d0b6d9d7-33fd-4d60-8483-d00cb06e6d58 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:24:37.048427 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b6d9d7-33fd-4d60-8483-d00cb06e6d58 service neutron] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:24:37.053785 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c24650-bf0b-4c64-bd2e-8f3043378cd8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cd171bf4df94f8fab397e42835feeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:24:37.054414 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c24650-bf0b-4c64-bd2e-8f3043378cd8 service nova] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:24:37.054696 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c24650-bf0b-4c64-bd2e-8f3043378cd8 service nova] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:24:37.054968 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c24650-bf0b-4c64-bd2e-8f3043378cd8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:24:37.056633 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-b5c24650-bf0b-4c64-bd2e-8f3043378cd8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:24:37.114603 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c24650-bf0b-4c64-bd2e-8f3043378cd8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:24:37.116184 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c24650-bf0b-4c64-bd2e-8f3043378cd8 service nova] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:24:37.125979 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1215/2416] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:24:37 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:24:44.413577 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=6 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:24:47.121886 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=34 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:24:49.164968 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f67965c-52fa-47cb-9e81-810f16a6b6df None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:24:49.167003 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f67965c-52fa-47cb-9e81-810f16a6b6df None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:24:49.167003 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f67965c-52fa-47cb-9e81-810f16a6b6df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:24:49.191014 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-7f67965c-52fa-47cb-9e81-810f16a6b6df None None] MFA Rules not processed for user `68b69b25b19a4addb9aff7fb7a710e56`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:24:49.192519 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-7f67965c-52fa-47cb-9e81-810f16a6b6df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:24:49.248591 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1202/2417] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:24:49 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:24:49.573472 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c24650-bf0b-4c64-bd2e-8f3043378cd8 service nova] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:24:49.583153 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c24650-bf0b-4c64-bd2e-8f3043378cd8 service nova] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:24:49.586740 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411d0c4e-e73e-44c6-b7ab-83761ec9b1c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:24:49.587351 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411d0c4e-e73e-44c6-b7ab-83761ec9b1c6 service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:24:49.587641 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411d0c4e-e73e-44c6-b7ab-83761ec9b1c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:24:49.587923 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411d0c4e-e73e-44c6-b7ab-83761ec9b1c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:24:49.589629 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-411d0c4e-e73e-44c6-b7ab-83761ec9b1c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:24:49.599793 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411d0c4e-e73e-44c6-b7ab-83761ec9b1c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:24:49.601278 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411d0c4e-e73e-44c6-b7ab-83761ec9b1c6 service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:24:49.609715 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1216/2418] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:24:49 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:24:59.241701 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=25 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:24:59.605579 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=6 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:25:00.556143 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f67965c-52fa-47cb-9e81-810f16a6b6df None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:25:00.570046 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f67965c-52fa-47cb-9e81-810f16a6b6df None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:25:00.575774 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c13e680-4078-4349-a699-dcdb13786d76 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cd171bf4df94f8fab397e42835feeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:25:00.576516 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c13e680-4078-4349-a699-dcdb13786d76 service nova] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:25:00.576783 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c13e680-4078-4349-a699-dcdb13786d76 service nova] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:25:00.577087 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c13e680-4078-4349-a699-dcdb13786d76 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:25:00.583882 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-1c13e680-4078-4349-a699-dcdb13786d76 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:25:00.596369 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c13e680-4078-4349-a699-dcdb13786d76 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:25:00.597949 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c13e680-4078-4349-a699-dcdb13786d76 service nova] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:25:00.609581 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1203/2419] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:25:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:25:05.231984 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411d0c4e-e73e-44c6-b7ab-83761ec9b1c6 service neutron] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:25:05.240725 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411d0c4e-e73e-44c6-b7ab-83761ec9b1c6 service neutron] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:25:05.244424 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbffe0d-a634-4a24-b111-96a2694b2869 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:25:05.245017 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbffe0d-a634-4a24-b111-96a2694b2869 service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:25:05.245295 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbffe0d-a634-4a24-b111-96a2694b2869 service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:25:05.245568 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbffe0d-a634-4a24-b111-96a2694b2869 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:25:05.255096 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbffe0d-a634-4a24-b111-96a2694b2869 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:25:05.256508 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbffe0d-a634-4a24-b111-96a2694b2869 service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:25:05.265996 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1217/2420] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:25:05 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:25:07.440126 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c13e680-4078-4349-a699-dcdb13786d76 service nova] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:25:07.450087 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c13e680-4078-4349-a699-dcdb13786d76 service nova] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:25:07.456297 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e391b8ee-06ef-4842-8210-ddb83d17d154 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cd171bf4df94f8fab397e42835feeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:25:07.456907 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e391b8ee-06ef-4842-8210-ddb83d17d154 service nova] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:25:07.457188 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e391b8ee-06ef-4842-8210-ddb83d17d154 service nova] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:25:07.457465 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e391b8ee-06ef-4842-8210-ddb83d17d154 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:25:07.459297 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-e391b8ee-06ef-4842-8210-ddb83d17d154 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:25:07.471765 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e391b8ee-06ef-4842-8210-ddb83d17d154 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:25:07.473313 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e391b8ee-06ef-4842-8210-ddb83d17d154 service nova] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:25:07.484397 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1204/2421] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:25:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 25 03:25:15.261782 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=6 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:25:17.480197 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=12 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:25:23.207721 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b056c1d-0aae-4359-8fd2-a7e2e6624bad None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:25:23.208285 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b056c1d-0aae-4359-8fd2-a7e2e6624bad None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:25:23.208285 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b056c1d-0aae-4359-8fd2-a7e2e6624bad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:25:23.227442 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-1b056c1d-0aae-4359-8fd2-a7e2e6624bad None None] MFA Rules not processed for user `c7dc5019b26a4a8abf6436be3f1e56af`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:25:23.228638 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-1b056c1d-0aae-4359-8fd2-a7e2e6624bad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:25:23.235900 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1218/2422] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:25:23 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:25:33.221646 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=3 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:26:16.583608 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669f870c-b6b6-414e-807f-d60d9fbac490 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:26:16.584151 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669f870c-b6b6-414e-807f-d60d9fbac490 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:26:16.584151 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669f870c-b6b6-414e-807f-d60d9fbac490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:26:16.602983 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-669f870c-b6b6-414e-807f-d60d9fbac490 None None] MFA Rules not processed for user `d0a0e9c327564a3db8b89ea706f0fb36`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:26:16.604163 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-669f870c-b6b6-414e-807f-d60d9fbac490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:26:16.665367 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1205/2423] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:26:16 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:26:26.657907 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=25 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:26:48.211153 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b056c1d-0aae-4359-8fd2-a7e2e6624bad None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:26:48.224625 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b056c1d-0aae-4359-8fd2-a7e2e6624bad None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:26:48.230586 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3270448c-644a-40c7-9976-e00fa9606e04 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cd171bf4df94f8fab397e42835feeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:26:48.231207 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3270448c-644a-40c7-9976-e00fa9606e04 service nova] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:26:48.231487 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3270448c-644a-40c7-9976-e00fa9606e04 service nova] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:26:48.231792 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3270448c-644a-40c7-9976-e00fa9606e04 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:26:48.233527 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-3270448c-644a-40c7-9976-e00fa9606e04 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:26:48.244508 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3270448c-644a-40c7-9976-e00fa9606e04 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:26:48.246121 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3270448c-644a-40c7-9976-e00fa9606e04 service nova] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:26:48.255666 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1219/2424] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:26:48 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:26:58.250998 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=6 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:27:08.048495 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669f870c-b6b6-414e-807f-d60d9fbac490 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:27:08.111626 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669f870c-b6b6-414e-807f-d60d9fbac490 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:27:08.117248 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caa0341-f71c-414e-b412-c6813583872b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b692edd3e8c41289cc42e1afb4a6dd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:27:08.117992 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caa0341-f71c-414e-b412-c6813583872b service placement] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:27:08.118288 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caa0341-f71c-414e-b412-c6813583872b service placement] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:27:08.118568 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caa0341-f71c-414e-b412-c6813583872b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:27:08.120308 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-9caa0341-f71c-414e-b412-c6813583872b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:27:08.155505 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caa0341-f71c-414e-b412-c6813583872b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:27:08.157350 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caa0341-f71c-414e-b412-c6813583872b service placement] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:27:08.168356 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1206/2425] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:27:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:27:08.534396 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3270448c-644a-40c7-9976-e00fa9606e04 service nova] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:27:08.562784 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3270448c-644a-40c7-9976-e00fa9606e04 service nova] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:27:08.562784 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd5fe09-037e-40e8-a205-35f05f393812 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:27:08.562784 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd5fe09-037e-40e8-a205-35f05f393812 service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:27:08.562784 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd5fe09-037e-40e8-a205-35f05f393812 service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:27:08.563596 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd5fe09-037e-40e8-a205-35f05f393812 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:27:08.563596 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-cbd5fe09-037e-40e8-a205-35f05f393812 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:27:08.654411 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd5fe09-037e-40e8-a205-35f05f393812 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:27:08.656024 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd5fe09-037e-40e8-a205-35f05f393812 service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:27:08.677600 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1220/2426] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:27:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:27:08.799632 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f8abce-4627-4dd4-a6fa-46930c0f0357 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:27:08.799923 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f8abce-4627-4dd4-a6fa-46930c0f0357 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:27:08.800204 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f8abce-4627-4dd4-a6fa-46930c0f0357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:27:08.820410 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-54f8abce-4627-4dd4-a6fa-46930c0f0357 None None] MFA Rules not processed for user `68b69b25b19a4addb9aff7fb7a710e56`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:27:08.824154 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-54f8abce-4627-4dd4-a6fa-46930c0f0357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:27:08.832110 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1207/2427] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:27:08 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:27:14.619939 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e242fe-918a-4162-ac2d-a7e15dcaa406 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:27:14.620458 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e242fe-918a-4162-ac2d-a7e15dcaa406 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:27:14.620977 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e242fe-918a-4162-ac2d-a7e15dcaa406 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:27:14.622776 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1221/2428] 10.222.0.58 () {54 vars in 921 bytes} [Thu Aug 25 03:27:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:27:14.655083 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee11368-f9eb-457d-8bc7-9c15648195e1 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:27:14.656243 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee11368-f9eb-457d-8bc7-9c15648195e1 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:27:14.656599 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee11368-f9eb-457d-8bc7-9c15648195e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:27:14.737519 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-6ee11368-f9eb-457d-8bc7-9c15648195e1 None None] MFA Rules not processed for user `2b692edd3e8c41289cc42e1afb4a6dd8`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:27:14.740518 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-6ee11368-f9eb-457d-8bc7-9c15648195e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:27:14.753564 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1208/2429] 10.222.0.58 () {58 vars in 1018 bytes} [Thu Aug 25 03:27:14 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:27:14.782938 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e242fe-918a-4162-ac2d-a7e15dcaa406 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:27:14.795297 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee11368-f9eb-457d-8bc7-9c15648195e1 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:27:14.797451 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-6ee11368-f9eb-457d-8bc7-9c15648195e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:27:14.801956 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e242fe-918a-4162-ac2d-a7e15dcaa406 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:27:14.812679 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818d8066-32dd-42c2-9c1c-cd6d52d5315f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b692edd3e8c41289cc42e1afb4a6dd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:27:14.812679 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818d8066-32dd-42c2-9c1c-cd6d52d5315f service placement] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:27:14.812679 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818d8066-32dd-42c2-9c1c-cd6d52d5315f service placement] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:27:14.812679 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818d8066-32dd-42c2-9c1c-cd6d52d5315f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:27:14.815143 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee11368-f9eb-457d-8bc7-9c15648195e1 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:27:14.821410 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb089d7-a3c5-4b07-bcdb-138516059759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:27:14.822178 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb089d7-a3c5-4b07-bcdb-138516059759 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:27:14.822636 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb089d7-a3c5-4b07-bcdb-138516059759 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:27:14.823009 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb089d7-a3c5-4b07-bcdb-138516059759 admin admin] PATH_INFO: `/v3/users/c7dc5019b26a4a8abf6436be3f1e56af` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:27:14.824925 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb089d7-a3c5-4b07-bcdb-138516059759 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7dc5019b26a4a8abf6436be3f1e56af)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:27:14.827067 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb089d7-a3c5-4b07-bcdb-138516059759 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:27:14.828147 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818d8066-32dd-42c2-9c1c-cd6d52d5315f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:27:14.830487 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818d8066-32dd-42c2-9c1c-cd6d52d5315f service placement] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:27:14.859425 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1222/2430] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:27:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:27:15.008171 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-9cb089d7-a3c5-4b07-bcdb-138516059759 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7dc5019b26a4a8abf6436be3f1e56af'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:27:15.066705 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-9cb089d7-a3c5-4b07-bcdb-138516059759 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7dc5019b26a4a8abf6436be3f1e56af'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:27:15.067465 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-9cb089d7-a3c5-4b07-bcdb-138516059759 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7dc5019b26a4a8abf6436be3f1e56af'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:27:15.068099 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-9cb089d7-a3c5-4b07-bcdb-138516059759 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7dc5019b26a4a8abf6436be3f1e56af'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:27:15.082745 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-9cb089d7-a3c5-4b07-bcdb-138516059759 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7dc5019b26a4a8abf6436be3f1e56af'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:27:15.099326 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1209/2431] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:27:14 2022] DELETE /identity/v3/users/c7dc5019b26a4a8abf6436be3f1e56af => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:27:15.496065 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818d8066-32dd-42c2-9c1c-cd6d52d5315f service placement] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:27:15.508086 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818d8066-32dd-42c2-9c1c-cd6d52d5315f service placement] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:27:15.569401 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdffedb-37fb-48f8-bf7e-4e95c7e5c68a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:27:15.570144 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdffedb-37fb-48f8-bf7e-4e95c7e5c68a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:27:15.570458 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdffedb-37fb-48f8-bf7e-4e95c7e5c68a admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:27:15.570733 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdffedb-37fb-48f8-bf7e-4e95c7e5c68a admin admin] PATH_INFO: `/v3/projects/d0928e519ec247b988a254d7918cbfed` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:27:15.571892 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdffedb-37fb-48f8-bf7e-4e95c7e5c68a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0928e519ec247b988a254d7918cbfed)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:27:15.574181 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdffedb-37fb-48f8-bf7e-4e95c7e5c68a admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:27:15.705798 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:27:15.709593 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-6bdffedb-37fb-48f8-bf7e-4e95c7e5c68a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0928e519ec247b988a254d7918cbfed', 'request_id': 'req-6bdffedb-37fb-48f8-bf7e-4e95c7e5c68a'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:27:15.769110 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-6bdffedb-37fb-48f8-bf7e-4e95c7e5c68a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0928e519ec247b988a254d7918cbfed', 'request_id': 'req-6bdffedb-37fb-48f8-bf7e-4e95c7e5c68a'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:27:15.775645 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-6bdffedb-37fb-48f8-bf7e-4e95c7e5c68a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0928e519ec247b988a254d7918cbfed', 'request_id': 'req-6bdffedb-37fb-48f8-bf7e-4e95c7e5c68a'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:27:15.787805 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-6bdffedb-37fb-48f8-bf7e-4e95c7e5c68a admin admin] The token cache is being invalidate because project d0928e519ec247b988a254d7918cbfed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:27:15.788325 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-6bdffedb-37fb-48f8-bf7e-4e95c7e5c68a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:27:15.789174 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-6bdffedb-37fb-48f8-bf7e-4e95c7e5c68a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:27:15.790688 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1223/2432] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:27:15 2022] DELETE /identity/v3/projects/d0928e519ec247b988a254d7918cbfed => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:27:15.795714 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d0bb77-0f7a-41ae-a628-46f1d73c3256 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:27:15.796197 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d0bb77-0f7a-41ae-a628-46f1d73c3256 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:27:15.796345 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d0bb77-0f7a-41ae-a628-46f1d73c3256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:27:15.810155 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:27:15.857776 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-57d0bb77-0f7a-41ae-a628-46f1d73c3256 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:27:15.859411 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-57d0bb77-0f7a-41ae-a628-46f1d73c3256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:27:15.934562 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1210/2433] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:27:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:27:15.944046 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7f6bd6-09a4-451f-b152-8ce2ac4f59a2 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:27:15.944330 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7f6bd6-09a4-451f-b152-8ce2ac4f59a2 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:27:15.944649 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7f6bd6-09a4-451f-b152-8ce2ac4f59a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:27:15.964254 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055a1a9a-7ace-4e13-bf6d-f3da04206712 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:27:15.964635 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055a1a9a-7ace-4e13-bf6d-f3da04206712 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:27:15.965336 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055a1a9a-7ace-4e13-bf6d-f3da04206712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:27:15.969762 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-0b7f6bd6-09a4-451f-b152-8ce2ac4f59a2 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:27:15.971877 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-0b7f6bd6-09a4-451f-b152-8ce2ac4f59a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:27:15.982890 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1224/2434] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:27:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:27:15.988684 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7f6bd6-09a4-451f-b152-8ce2ac4f59a2 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:27:15.989760 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-055a1a9a-7ace-4e13-bf6d-f3da04206712 None None] MFA Rules not processed for user `83e90afadb164c82bc93c47099c14460`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:27:15.994302 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-055a1a9a-7ace-4e13-bf6d-f3da04206712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:27:16.004549 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7f6bd6-09a4-451f-b152-8ce2ac4f59a2 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:27:16.011496 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a834b3e5-477c-4cc7-b970-ad3997dff80a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:27:16.012116 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a834b3e5-477c-4cc7-b970-ad3997dff80a admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:27:16.012535 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a834b3e5-477c-4cc7-b970-ad3997dff80a admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:27:16.012821 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a834b3e5-477c-4cc7-b970-ad3997dff80a admin admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:27:16.013853 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a834b3e5-477c-4cc7-b970-ad3997dff80a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:27:16.014167 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a834b3e5-477c-4cc7-b970-ad3997dff80a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:27:16.016046 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a834b3e5-477c-4cc7-b970-ad3997dff80a admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:27:16.031406 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1225/2435] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Aug 25 03:27:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:27:16.038082 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a834b3e5-477c-4cc7-b970-ad3997dff80a admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:27:16.057354 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a834b3e5-477c-4cc7-b970-ad3997dff80a admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:27:16.062405 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ca2ddc-a184-432c-a43a-7a67657cb413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:27:16.066711 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ca2ddc-a184-432c-a43a-7a67657cb413 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:27:16.066711 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ca2ddc-a184-432c-a43a-7a67657cb413 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:27:16.066711 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ca2ddc-a184-432c-a43a-7a67657cb413 admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:27:16.066711 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ca2ddc-a184-432c-a43a-7a67657cb413 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:27:16.066711 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ca2ddc-a184-432c-a43a-7a67657cb413 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:27:16.079073 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1211/2436] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:27:15 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:27:16.180918 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1226/2437] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:27:16 2022] POST /identity/v3/projects => generated 387 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:27:16.181807 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055a1a9a-7ace-4e13-bf6d-f3da04206712 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:27:16.192775 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055a1a9a-7ace-4e13-bf6d-f3da04206712 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:27:16.251054 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dec18b-86e3-41c6-8cc3-2950d3435566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:27:16.251670 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dec18b-86e3-41c6-8cc3-2950d3435566 admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:27:16.251950 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dec18b-86e3-41c6-8cc3-2950d3435566 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:27:16.252283 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dec18b-86e3-41c6-8cc3-2950d3435566 admin admin] PATH_INFO: `/v3/users` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:27:16.252908 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dec18b-86e3-41c6-8cc3-2950d3435566 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:27:16.254860 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dec18b-86e3-41c6-8cc3-2950d3435566 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:27:16.358534 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1212/2438] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:27:16 2022] POST /identity/v3/users => generated 294 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:27:16.362635 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ca2ddc-a184-432c-a43a-7a67657cb413 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:27:16.374225 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ca2ddc-a184-432c-a43a-7a67657cb413 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:27:16.379204 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c47d19-dd41-4023-a9d6-0fcd32cc31a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:27:16.379865 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c47d19-dd41-4023-a9d6-0fcd32cc31a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:27:16.380149 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c47d19-dd41-4023-a9d6-0fcd32cc31a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:27:16.380441 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c47d19-dd41-4023-a9d6-0fcd32cc31a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:27:16.381283 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c47d19-dd41-4023-a9d6-0fcd32cc31a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:27:16.382937 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c47d19-dd41-4023-a9d6-0fcd32cc31a2 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:27:16.396319 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1227/2439] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:27:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 25 03:27:16.397679 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dec18b-86e3-41c6-8cc3-2950d3435566 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:27:16.408222 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dec18b-86e3-41c6-8cc3-2950d3435566 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:27:16.413685 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca9ef25-1239-4ca3-b0d8-770418d0d53a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:27:16.414322 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca9ef25-1239-4ca3-b0d8-770418d0d53a admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:27:16.414637 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca9ef25-1239-4ca3-b0d8-770418d0d53a admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:27:16.414915 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca9ef25-1239-4ca3-b0d8-770418d0d53a admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:27:16.415649 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca9ef25-1239-4ca3-b0d8-770418d0d53a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:27:16.417185 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca9ef25-1239-4ca3-b0d8-770418d0d53a admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:27:16.431690 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1213/2440] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:27:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:27:16.432755 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c47d19-dd41-4023-a9d6-0fcd32cc31a2 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:27:16.443173 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c47d19-dd41-4023-a9d6-0fcd32cc31a2 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:27:16.447796 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca1a8db-f056-4aa5-b432-4bd911e83008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:27:16.448447 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca1a8db-f056-4aa5-b432-4bd911e83008 admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:27:16.448724 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca1a8db-f056-4aa5-b432-4bd911e83008 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:27:16.448995 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca1a8db-f056-4aa5-b432-4bd911e83008 admin admin] PATH_INFO: `/v3/projects/3e22a25921ea45eba0f29a87edaba447/users/08a52103c0f4467c8366bd88e8b655e9/roles/3255f77192524a16bc405173106871fc` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:27:16.464354 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca1a8db-f056-4aa5-b432-4bd911e83008 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e22a25921ea45eba0f29a87edaba447, user_id=08a52103c0f4467c8366bd88e8b655e9, role_id=3255f77192524a16bc405173106871fc)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:27:16.467062 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca1a8db-f056-4aa5-b432-4bd911e83008 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:27:16.541309 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1228/2441] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:27:16 2022] PUT /identity/v3/projects/3e22a25921ea45eba0f29a87edaba447/users/08a52103c0f4467c8366bd88e8b655e9/roles/3255f77192524a16bc405173106871fc => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:27:16.546702 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a89ad9b-6389-4770-b715-296f7b86f3d2 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:27:16.546989 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a89ad9b-6389-4770-b715-296f7b86f3d2 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:27:16.547534 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a89ad9b-6389-4770-b715-296f7b86f3d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:27:16.567599 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-1a89ad9b-6389-4770-b715-296f7b86f3d2 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:27:16.568843 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-1a89ad9b-6389-4770-b715-296f7b86f3d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:27:16.573920 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:27:16.636130 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1214/2442] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:27:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:27:16.645605 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca1a8db-f056-4aa5-b432-4bd911e83008 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:27:16.647857 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-0ca1a8db-f056-4aa5-b432-4bd911e83008 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:27:16.733434 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca1a8db-f056-4aa5-b432-4bd911e83008 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:27:16.737912 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d07b58-533c-4da5-96ae-c8efd8144f10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:27:16.738803 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d07b58-533c-4da5-96ae-c8efd8144f10 service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:27:16.739262 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d07b58-533c-4da5-96ae-c8efd8144f10 service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:27:16.740338 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d07b58-533c-4da5-96ae-c8efd8144f10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:27:16.749821 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d07b58-533c-4da5-96ae-c8efd8144f10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:27:16.751349 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d07b58-533c-4da5-96ae-c8efd8144f10 service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:27:16.761086 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1229/2443] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:27:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:27:24.136002 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9921a92-c3e3-48e2-9036-2dc39db0d1f8 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:27:24.137148 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9921a92-c3e3-48e2-9036-2dc39db0d1f8 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:27:24.137440 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9921a92-c3e3-48e2-9036-2dc39db0d1f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:27:24.184779 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-f9921a92-c3e3-48e2-9036-2dc39db0d1f8 None None] MFA Rules not processed for user `08a52103c0f4467c8366bd88e8b655e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:27:24.186587 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-f9921a92-c3e3-48e2-9036-2dc39db0d1f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:27:24.277112 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1215/2444] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:27:24 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:27:24.294078 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d07b58-533c-4da5-96ae-c8efd8144f10 service neutron] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:27:24.295584 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-b6d07b58-533c-4da5-96ae-c8efd8144f10 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:27:24.342691 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:27:24.439201 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d07b58-533c-4da5-96ae-c8efd8144f10 service neutron] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:27:24.451080 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ef0b27-d01c-42e9-839f-2732c5902265 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cd171bf4df94f8fab397e42835feeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:27:24.452440 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ef0b27-d01c-42e9-839f-2732c5902265 service nova] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:27:24.452932 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ef0b27-d01c-42e9-839f-2732c5902265 service nova] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:27:24.453861 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ef0b27-d01c-42e9-839f-2732c5902265 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:27:24.467594 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ef0b27-d01c-42e9-839f-2732c5902265 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:27:24.470467 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ef0b27-d01c-42e9-839f-2732c5902265 service nova] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:27:24.503386 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1230/2445] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:27:24 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:27:26.975545 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb686a94-29c9-4711-b780-adb1fa84cfb9 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:27:26.976023 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb686a94-29c9-4711-b780-adb1fa84cfb9 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:27:26.976123 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb686a94-29c9-4711-b780-adb1fa84cfb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:27:26.995071 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-eb686a94-29c9-4711-b780-adb1fa84cfb9 None None] MFA Rules not processed for user `08a52103c0f4467c8366bd88e8b655e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:27:26.996246 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-eb686a94-29c9-4711-b780-adb1fa84cfb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:27:27.003077 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1216/2446] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:27:26 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:27:34.545929 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=29 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:27:36.989969 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=57 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:27:55.926176 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a9ac6c-80c0-4337-a51d-4db33248a6be None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:27:55.926695 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a9ac6c-80c0-4337-a51d-4db33248a6be None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:27:55.926872 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a9ac6c-80c0-4337-a51d-4db33248a6be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:27:55.946947 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-e3a9ac6c-80c0-4337-a51d-4db33248a6be None None] MFA Rules not processed for user `712c35b765b64919aa3eaaf02892590d`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:27:55.948169 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-e3a9ac6c-80c0-4337-a51d-4db33248a6be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:27:56.004788 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1231/2447] 10.222.0.58 () {58 vars in 967 bytes} [Thu Aug 25 03:27:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:27:56.045619 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb686a94-29c9-4711-b780-adb1fa84cfb9 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:27:56.047165 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-eb686a94-29c9-4711-b780-adb1fa84cfb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:27:56.115625 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb686a94-29c9-4711-b780-adb1fa84cfb9 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:27:56.120610 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6129830-7218-4dc6-9b62-e639a3de9e93 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5047e1a7cd2d416d90b2d8c5a6596522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:27:56.121210 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6129830-7218-4dc6-9b62-e639a3de9e93 service swift] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:27:56.121490 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6129830-7218-4dc6-9b62-e639a3de9e93 service swift] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:27:56.121763 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6129830-7218-4dc6-9b62-e639a3de9e93 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:27:56.129560 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6129830-7218-4dc6-9b62-e639a3de9e93 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:27:56.131181 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6129830-7218-4dc6-9b62-e639a3de9e93 service swift] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:27:56.138941 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1217/2448] 10.222.0.58 () {60 vars in 1384 bytes} [Thu Aug 25 03:27:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 25 03:28:04.209864 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a9ac6c-80c0-4337-a51d-4db33248a6be None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:28:04.212108 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-e3a9ac6c-80c0-4337-a51d-4db33248a6be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:28:04.283083 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a9ac6c-80c0-4337-a51d-4db33248a6be None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:28:04.288418 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94e45ec-46cc-4bba-9f78-fb8b0d7dee72 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b692edd3e8c41289cc42e1afb4a6dd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:28:04.292188 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94e45ec-46cc-4bba-9f78-fb8b0d7dee72 service placement] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:28:04.292188 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94e45ec-46cc-4bba-9f78-fb8b0d7dee72 service placement] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:28:04.292188 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94e45ec-46cc-4bba-9f78-fb8b0d7dee72 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:28:04.292188 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-b94e45ec-46cc-4bba-9f78-fb8b0d7dee72 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:28:04.304340 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94e45ec-46cc-4bba-9f78-fb8b0d7dee72 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:28:04.305919 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94e45ec-46cc-4bba-9f78-fb8b0d7dee72 service placement] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:28:04.316499 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1232/2449] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:28:04 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:28:04.398757 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6129830-7218-4dc6-9b62-e639a3de9e93 service swift] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:28:04.400261 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-b6129830-7218-4dc6-9b62-e639a3de9e93 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:28:04.414266 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6129830-7218-4dc6-9b62-e639a3de9e93 service swift] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:28:04.420539 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b3bf01-7ef1-443e-bde1-fb6ca6cb808f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:28:04.421177 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b3bf01-7ef1-443e-bde1-fb6ca6cb808f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:28:04.421483 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b3bf01-7ef1-443e-bde1-fb6ca6cb808f admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:28:04.421908 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b3bf01-7ef1-443e-bde1-fb6ca6cb808f admin admin] PATH_INFO: `/v3/users/dd2985023e274f19a868da37aa791859` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:28:04.423059 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b3bf01-7ef1-443e-bde1-fb6ca6cb808f admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd2985023e274f19a868da37aa791859)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:28:04.425076 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b3bf01-7ef1-443e-bde1-fb6ca6cb808f admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:28:04.610134 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-f1b3bf01-7ef1-443e-bde1-fb6ca6cb808f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd2985023e274f19a868da37aa791859'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:28:04.682410 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-f1b3bf01-7ef1-443e-bde1-fb6ca6cb808f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd2985023e274f19a868da37aa791859'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:28:04.683040 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-f1b3bf01-7ef1-443e-bde1-fb6ca6cb808f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd2985023e274f19a868da37aa791859'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:28:04.683644 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-f1b3bf01-7ef1-443e-bde1-fb6ca6cb808f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd2985023e274f19a868da37aa791859'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:28:04.697119 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-f1b3bf01-7ef1-443e-bde1-fb6ca6cb808f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd2985023e274f19a868da37aa791859'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:28:04.709173 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1218/2450] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:28:04 2022] DELETE /identity/v3/users/dd2985023e274f19a868da37aa791859 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:28:04.946151 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94e45ec-46cc-4bba-9f78-fb8b0d7dee72 service placement] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:28:04.958876 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94e45ec-46cc-4bba-9f78-fb8b0d7dee72 service placement] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:28:05.019202 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff3f25c-769e-4d7c-81e9-ec7b6d26384c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:28:05.019835 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff3f25c-769e-4d7c-81e9-ec7b6d26384c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:28:05.020118 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff3f25c-769e-4d7c-81e9-ec7b6d26384c admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:28:05.020394 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff3f25c-769e-4d7c-81e9-ec7b6d26384c admin admin] PATH_INFO: `/v3/projects/3e91375fe4114bf98447ccfcb6f8475f` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:28:05.021501 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff3f25c-769e-4d7c-81e9-ec7b6d26384c admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e91375fe4114bf98447ccfcb6f8475f)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:28:05.023624 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff3f25c-769e-4d7c-81e9-ec7b6d26384c admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:28:05.136140 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:28:05.144226 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-8ff3f25c-769e-4d7c-81e9-ec7b6d26384c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e91375fe4114bf98447ccfcb6f8475f', 'request_id': 'req-8ff3f25c-769e-4d7c-81e9-ec7b6d26384c'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:28:05.241715 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-8ff3f25c-769e-4d7c-81e9-ec7b6d26384c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e91375fe4114bf98447ccfcb6f8475f', 'request_id': 'req-8ff3f25c-769e-4d7c-81e9-ec7b6d26384c'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:28:05.247304 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-8ff3f25c-769e-4d7c-81e9-ec7b6d26384c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e91375fe4114bf98447ccfcb6f8475f', 'request_id': 'req-8ff3f25c-769e-4d7c-81e9-ec7b6d26384c'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:28:05.258626 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-8ff3f25c-769e-4d7c-81e9-ec7b6d26384c admin admin] The token cache is being invalidate because project 3e91375fe4114bf98447ccfcb6f8475f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:28:05.258916 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-8ff3f25c-769e-4d7c-81e9-ec7b6d26384c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:28:05.259552 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-8ff3f25c-769e-4d7c-81e9-ec7b6d26384c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:28:05.261039 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1233/2451] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:28:04 2022] DELETE /identity/v3/projects/3e91375fe4114bf98447ccfcb6f8475f => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:28:05.265062 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b3bf01-7ef1-443e-bde1-fb6ca6cb808f admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:28:05.266522 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-f1b3bf01-7ef1-443e-bde1-fb6ca6cb808f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:28:05.279919 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b3bf01-7ef1-443e-bde1-fb6ca6cb808f admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:28:05.334623 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e471e32d-6063-4e70-927f-418ade054cba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:28:05.335248 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e471e32d-6063-4e70-927f-418ade054cba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:28:05.336019 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e471e32d-6063-4e70-927f-418ade054cba admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:28:05.336296 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e471e32d-6063-4e70-927f-418ade054cba admin admin] PATH_INFO: `/v3/users/f7f83c84f8814829aa7114ad74c50232` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:28:05.337507 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e471e32d-6063-4e70-927f-418ade054cba admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7f83c84f8814829aa7114ad74c50232)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:28:05.339486 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e471e32d-6063-4e70-927f-418ade054cba admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:28:05.341794 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:28:05.518337 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-e471e32d-6063-4e70-927f-418ade054cba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7f83c84f8814829aa7114ad74c50232'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:28:05.590464 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-e471e32d-6063-4e70-927f-418ade054cba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7f83c84f8814829aa7114ad74c50232'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:28:05.591085 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-e471e32d-6063-4e70-927f-418ade054cba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7f83c84f8814829aa7114ad74c50232'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:28:05.591705 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-e471e32d-6063-4e70-927f-418ade054cba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7f83c84f8814829aa7114ad74c50232'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:28:05.604311 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-e471e32d-6063-4e70-927f-418ade054cba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7f83c84f8814829aa7114ad74c50232'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:28:05.616301 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1219/2452] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:28:05 2022] DELETE /identity/v3/users/f7f83c84f8814829aa7114ad74c50232 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:28:05.837870 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff3f25c-769e-4d7c-81e9-ec7b6d26384c admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:28:05.839349 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-8ff3f25c-769e-4d7c-81e9-ec7b6d26384c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:28:05.851838 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff3f25c-769e-4d7c-81e9-ec7b6d26384c admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:28:05.909881 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdab17e3-87f1-4517-8e01-738da8d8d355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:28:05.910664 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdab17e3-87f1-4517-8e01-738da8d8d355 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:28:05.910944 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdab17e3-87f1-4517-8e01-738da8d8d355 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:28:05.911215 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdab17e3-87f1-4517-8e01-738da8d8d355 admin admin] PATH_INFO: `/v3/projects/d4cbf559385341bd9cd2fa5388a2c7bb` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:28:05.912373 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdab17e3-87f1-4517-8e01-738da8d8d355 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4cbf559385341bd9cd2fa5388a2c7bb)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:28:05.914357 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdab17e3-87f1-4517-8e01-738da8d8d355 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:28:06.018457 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-bdab17e3-87f1-4517-8e01-738da8d8d355 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4cbf559385341bd9cd2fa5388a2c7bb', 'request_id': 'req-bdab17e3-87f1-4517-8e01-738da8d8d355'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:28:06.110475 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-bdab17e3-87f1-4517-8e01-738da8d8d355 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4cbf559385341bd9cd2fa5388a2c7bb', 'request_id': 'req-bdab17e3-87f1-4517-8e01-738da8d8d355'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:28:06.116108 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-bdab17e3-87f1-4517-8e01-738da8d8d355 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4cbf559385341bd9cd2fa5388a2c7bb', 'request_id': 'req-bdab17e3-87f1-4517-8e01-738da8d8d355'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:28:06.127851 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-bdab17e3-87f1-4517-8e01-738da8d8d355 admin admin] The token cache is being invalidate because project d4cbf559385341bd9cd2fa5388a2c7bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:28:06.128143 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-bdab17e3-87f1-4517-8e01-738da8d8d355 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:28:06.128775 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-bdab17e3-87f1-4517-8e01-738da8d8d355 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:28:06.130248 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1234/2453] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:28:05 2022] DELETE /identity/v3/projects/d4cbf559385341bd9cd2fa5388a2c7bb => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:28:15.612798 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=25,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:28:16.124452 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=62,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:28:56.610596 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e471e32d-6063-4e70-927f-418ade054cba admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:28:56.612002 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-e471e32d-6063-4e70-927f-418ade054cba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:28:56.691206 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e471e32d-6063-4e70-927f-418ade054cba admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:28:56.738993 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaafb2ac-734a-4354-9b94-cc7fc861a9fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60193ba9e1c14df3bc9eebcc6f28f2d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:28:56.739600 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaafb2ac-734a-4354-9b94-cc7fc861a9fb service cinder] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:28:56.739883 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaafb2ac-734a-4354-9b94-cc7fc861a9fb service cinder] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:28:56.740156 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaafb2ac-734a-4354-9b94-cc7fc861a9fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:28:56.742145 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-aaafb2ac-734a-4354-9b94-cc7fc861a9fb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:28:56.800531 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaafb2ac-734a-4354-9b94-cc7fc861a9fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:28:56.801966 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaafb2ac-734a-4354-9b94-cc7fc861a9fb service cinder] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:28:56.810808 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1220/2454] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:28:56 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:29:03.545539 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdab17e3-87f1-4517-8e01-738da8d8d355 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:29:03.546987 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-bdab17e3-87f1-4517-8e01-738da8d8d355 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:29:03.559687 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdab17e3-87f1-4517-8e01-738da8d8d355 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:29:03.563451 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846a35ad-9797-4d79-b002-b85f68d076d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60193ba9e1c14df3bc9eebcc6f28f2d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:29:03.564041 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846a35ad-9797-4d79-b002-b85f68d076d8 service cinder] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:29:03.564317 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846a35ad-9797-4d79-b002-b85f68d076d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:29:03.564615 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846a35ad-9797-4d79-b002-b85f68d076d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:29:03.566449 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-846a35ad-9797-4d79-b002-b85f68d076d8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:29:03.636780 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846a35ad-9797-4d79-b002-b85f68d076d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:29:03.638924 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846a35ad-9797-4d79-b002-b85f68d076d8 service cinder] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:29:03.649831 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1235/2455] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:29:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 25 03:29:06.806753 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=77 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:29:09.487654 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9b6589-7b0e-43a1-adaf-6bd5d7903e22 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:29:09.488326 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9b6589-7b0e-43a1-adaf-6bd5d7903e22 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:29:09.488326 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9b6589-7b0e-43a1-adaf-6bd5d7903e22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:29:09.507380 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-ef9b6589-7b0e-43a1-adaf-6bd5d7903e22 None None] MFA Rules not processed for user `d0a0e9c327564a3db8b89ea706f0fb36`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:29:09.508591 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-ef9b6589-7b0e-43a1-adaf-6bd5d7903e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:29:09.515977 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1221/2456] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:29:09 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:29:09.553333 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846a35ad-9797-4d79-b002-b85f68d076d8 service cinder] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:29:09.554775 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-846a35ad-9797-4d79-b002-b85f68d076d8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:29:09.566117 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846a35ad-9797-4d79-b002-b85f68d076d8 service cinder] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:29:09.569736 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c19a34b-a788-47a7-8773-eda6c92588b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60193ba9e1c14df3bc9eebcc6f28f2d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:29:09.570362 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c19a34b-a788-47a7-8773-eda6c92588b5 service cinder] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:29:09.570644 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c19a34b-a788-47a7-8773-eda6c92588b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:29:09.570917 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c19a34b-a788-47a7-8773-eda6c92588b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:29:09.572575 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-2c19a34b-a788-47a7-8773-eda6c92588b5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:29:09.586709 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c19a34b-a788-47a7-8773-eda6c92588b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:29:09.588171 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c19a34b-a788-47a7-8773-eda6c92588b5 service cinder] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:29:09.599004 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1236/2457] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:29:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:29:18.801492 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9b6589-7b0e-43a1-adaf-6bd5d7903e22 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:29:18.802923 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-ef9b6589-7b0e-43a1-adaf-6bd5d7903e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:29:18.819428 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9b6589-7b0e-43a1-adaf-6bd5d7903e22 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:29:18.825546 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23483c4-1177-4a21-948b-4ed7fcb80af4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cd171bf4df94f8fab397e42835feeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:29:18.826150 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23483c4-1177-4a21-948b-4ed7fcb80af4 service nova] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:29:18.826478 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23483c4-1177-4a21-948b-4ed7fcb80af4 service nova] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:29:18.826754 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23483c4-1177-4a21-948b-4ed7fcb80af4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:29:18.828423 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-f23483c4-1177-4a21-948b-4ed7fcb80af4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:29:18.907146 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23483c4-1177-4a21-948b-4ed7fcb80af4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:29:18.909640 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23483c4-1177-4a21-948b-4ed7fcb80af4 service nova] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:29:18.934938 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1222/2458] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:29:18 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:29:19.594268 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=40 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:29:28.930530 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=44 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:29:32.420045 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c19a34b-a788-47a7-8773-eda6c92588b5 service cinder] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:29:32.421586 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-2c19a34b-a788-47a7-8773-eda6c92588b5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:29:32.514582 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c19a34b-a788-47a7-8773-eda6c92588b5 service cinder] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:29:32.514582 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6e5b46-1e00-42df-93f9-6009749da43e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:29:32.514582 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6e5b46-1e00-42df-93f9-6009749da43e service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:29:32.514582 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6e5b46-1e00-42df-93f9-6009749da43e service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:29:32.515623 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6e5b46-1e00-42df-93f9-6009749da43e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:29:32.515623 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-de6e5b46-1e00-42df-93f9-6009749da43e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:29:32.576715 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6e5b46-1e00-42df-93f9-6009749da43e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:29:32.578645 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6e5b46-1e00-42df-93f9-6009749da43e service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:29:32.592207 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1237/2459] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:29:32 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:29:33.295679 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2c6ac0-75e8-4a95-a276-782e91338d91 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:29:33.296239 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2c6ac0-75e8-4a95-a276-782e91338d91 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:29:33.296354 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2c6ac0-75e8-4a95-a276-782e91338d91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:29:33.316687 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-3d2c6ac0-75e8-4a95-a276-782e91338d91 None None] MFA Rules not processed for user `68b69b25b19a4addb9aff7fb7a710e56`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:29:33.320020 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-3d2c6ac0-75e8-4a95-a276-782e91338d91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:29:33.379978 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1223/2460] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:29:33 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:29:34.077328 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6e5b46-1e00-42df-93f9-6009749da43e service neutron] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:29:34.078925 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-de6e5b46-1e00-42df-93f9-6009749da43e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:29:34.160026 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6e5b46-1e00-42df-93f9-6009749da43e service neutron] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:29:34.165340 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6a2b59-f054-4df0-b3f2-eaf3ed4efd6c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b692edd3e8c41289cc42e1afb4a6dd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:29:34.166002 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6a2b59-f054-4df0-b3f2-eaf3ed4efd6c service placement] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:29:34.167132 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6a2b59-f054-4df0-b3f2-eaf3ed4efd6c service placement] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:29:34.167132 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6a2b59-f054-4df0-b3f2-eaf3ed4efd6c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:29:34.168395 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-0c6a2b59-f054-4df0-b3f2-eaf3ed4efd6c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:29:34.180429 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6a2b59-f054-4df0-b3f2-eaf3ed4efd6c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:29:34.182162 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6a2b59-f054-4df0-b3f2-eaf3ed4efd6c service placement] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:29:34.193960 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1238/2461] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:29:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:29:34.841015 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2c6ac0-75e8-4a95-a276-782e91338d91 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:29:34.850867 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2c6ac0-75e8-4a95-a276-782e91338d91 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:29:34.854395 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2170a515-a396-402d-a3a9-567db1d174a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:29:34.854997 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2170a515-a396-402d-a3a9-567db1d174a3 service neutron] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:29:34.855276 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2170a515-a396-402d-a3a9-567db1d174a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:29:34.855581 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2170a515-a396-402d-a3a9-567db1d174a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:29:34.858056 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-2170a515-a396-402d-a3a9-567db1d174a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:29:34.869246 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2170a515-a396-402d-a3a9-567db1d174a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:29:34.870970 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2170a515-a396-402d-a3a9-567db1d174a3 service neutron] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:29:34.882341 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1224/2462] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:29:34 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:29:44.189825 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=82 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:29:44.877785 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=31 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:29:52.554769 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67197b08-1113-45bd-b7bc-e79231c8d299 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:29:52.555297 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67197b08-1113-45bd-b7bc-e79231c8d299 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:29:52.555418 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67197b08-1113-45bd-b7bc-e79231c8d299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:29:52.609280 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-67197b08-1113-45bd-b7bc-e79231c8d299 None None] MFA Rules not processed for user `712c35b765b64919aa3eaaf02892590d`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:29:52.610617 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-67197b08-1113-45bd-b7bc-e79231c8d299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:29:52.731755 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1239/2463] 10.222.0.58 () {58 vars in 1009 bytes} [Thu Aug 25 03:29:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:29:52.850761 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2170a515-a396-402d-a3a9-567db1d174a3 service neutron] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:29:52.853059 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-2170a515-a396-402d-a3a9-567db1d174a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:29:52.984391 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2170a515-a396-402d-a3a9-567db1d174a3 service neutron] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:29:52.992726 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef45ce1a-1064-4a8d-aaa2-af2091aeaf15 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5047e1a7cd2d416d90b2d8c5a6596522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:29:52.993983 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef45ce1a-1064-4a8d-aaa2-af2091aeaf15 service swift] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:29:52.994262 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef45ce1a-1064-4a8d-aaa2-af2091aeaf15 service swift] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:29:52.994830 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef45ce1a-1064-4a8d-aaa2-af2091aeaf15 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:29:53.019446 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef45ce1a-1064-4a8d-aaa2-af2091aeaf15 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:29:53.021779 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef45ce1a-1064-4a8d-aaa2-af2091aeaf15 service swift] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:29:53.047356 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1225/2464] 10.222.0.58 () {60 vars in 1384 bytes} [Thu Aug 25 03:29:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 25 03:29:53.056476 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67197b08-1113-45bd-b7bc-e79231c8d299 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:29:53.057948 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-67197b08-1113-45bd-b7bc-e79231c8d299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:29:53.081262 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67197b08-1113-45bd-b7bc-e79231c8d299 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:29:53.088409 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59acfad-7cfc-41af-875b-23199afc69db service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cd171bf4df94f8fab397e42835feeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:29:53.089398 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59acfad-7cfc-41af-875b-23199afc69db service nova] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:29:53.089833 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59acfad-7cfc-41af-875b-23199afc69db service nova] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:29:53.090415 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59acfad-7cfc-41af-875b-23199afc69db service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:29:53.093438 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-b59acfad-7cfc-41af-875b-23199afc69db service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:29:53.108109 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59acfad-7cfc-41af-875b-23199afc69db service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:29:53.109974 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59acfad-7cfc-41af-875b-23199afc69db service nova] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:29:53.129169 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1240/2465] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:29:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:30:00.063229 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef45ce1a-1064-4a8d-aaa2-af2091aeaf15 service swift] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:30:00.064654 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-ef45ce1a-1064-4a8d-aaa2-af2091aeaf15 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:30:00.077043 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef45ce1a-1064-4a8d-aaa2-af2091aeaf15 service swift] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:30:00.080688 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245c7159-a9b2-403d-864e-5fc647627771 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:30:00.081304 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245c7159-a9b2-403d-864e-5fc647627771 service neutron] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:00.081565 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245c7159-a9b2-403d-864e-5fc647627771 service neutron] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:00.081837 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245c7159-a9b2-403d-864e-5fc647627771 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:00.083537 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-245c7159-a9b2-403d-864e-5fc647627771 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:30:00.168160 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245c7159-a9b2-403d-864e-5fc647627771 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:30:00.169668 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245c7159-a9b2-403d-864e-5fc647627771 service neutron] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:30:00.179985 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1226/2466] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:30:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:30:03.119824 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=46 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:30:04.870913 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59acfad-7cfc-41af-875b-23199afc69db service nova] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:30:04.873385 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-b59acfad-7cfc-41af-875b-23199afc69db service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:30:04.895800 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59acfad-7cfc-41af-875b-23199afc69db service nova] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:30:04.900707 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d148df2-8dbb-431b-b53a-4635a76e96cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:30:04.901424 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d148df2-8dbb-431b-b53a-4635a76e96cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:04.901707 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d148df2-8dbb-431b-b53a-4635a76e96cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:04.902025 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d148df2-8dbb-431b-b53a-4635a76e96cf admin admin] PATH_INFO: `/v3/users/83e90afadb164c82bc93c47099c14460` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:04.903182 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d148df2-8dbb-431b-b53a-4635a76e96cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=83e90afadb164c82bc93c47099c14460)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:30:04.905634 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d148df2-8dbb-431b-b53a-4635a76e96cf admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:30:05.133064 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-3d148df2-8dbb-431b-b53a-4635a76e96cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83e90afadb164c82bc93c47099c14460'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:30:05.207384 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-3d148df2-8dbb-431b-b53a-4635a76e96cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83e90afadb164c82bc93c47099c14460'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:30:05.207941 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-3d148df2-8dbb-431b-b53a-4635a76e96cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83e90afadb164c82bc93c47099c14460'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:30:05.224537 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-3d148df2-8dbb-431b-b53a-4635a76e96cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83e90afadb164c82bc93c47099c14460'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:30:05.244233 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-3d148df2-8dbb-431b-b53a-4635a76e96cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83e90afadb164c82bc93c47099c14460'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:30:05.256287 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1241/2467] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:30:04 2022] DELETE /identity/v3/users/83e90afadb164c82bc93c47099c14460 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:30:05.613732 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245c7159-a9b2-403d-864e-5fc647627771 service neutron] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:30:05.624348 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245c7159-a9b2-403d-864e-5fc647627771 service neutron] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:30:05.684728 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a9e7df-0b3d-4bda-8195-d8fa5348af00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:30:05.686175 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a9e7df-0b3d-4bda-8195-d8fa5348af00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:05.686487 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a9e7df-0b3d-4bda-8195-d8fa5348af00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:05.686761 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a9e7df-0b3d-4bda-8195-d8fa5348af00 admin admin] PATH_INFO: `/v3/projects/d8e6240beed94fc38a7f1b85c7fad39c` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:05.687890 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a9e7df-0b3d-4bda-8195-d8fa5348af00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8e6240beed94fc38a7f1b85c7fad39c)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:30:05.689861 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a9e7df-0b3d-4bda-8195-d8fa5348af00 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:30:05.696344 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:30:05.840253 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-c3a9e7df-0b3d-4bda-8195-d8fa5348af00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8e6240beed94fc38a7f1b85c7fad39c', 'request_id': 'req-c3a9e7df-0b3d-4bda-8195-d8fa5348af00'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:30:05.907791 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-c3a9e7df-0b3d-4bda-8195-d8fa5348af00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8e6240beed94fc38a7f1b85c7fad39c', 'request_id': 'req-c3a9e7df-0b3d-4bda-8195-d8fa5348af00'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:30:05.913808 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-c3a9e7df-0b3d-4bda-8195-d8fa5348af00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8e6240beed94fc38a7f1b85c7fad39c', 'request_id': 'req-c3a9e7df-0b3d-4bda-8195-d8fa5348af00'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:30:05.925512 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-c3a9e7df-0b3d-4bda-8195-d8fa5348af00 admin admin] The token cache is being invalidate because project d8e6240beed94fc38a7f1b85c7fad39c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81995) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:30:05.925801 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-c3a9e7df-0b3d-4bda-8195-d8fa5348af00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:30:05.926471 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-c3a9e7df-0b3d-4bda-8195-d8fa5348af00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:30:05.927962 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1227/2468] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:30:05 2022] DELETE /identity/v3/projects/d8e6240beed94fc38a7f1b85c7fad39c => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:30:05.932974 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba0f937-d3bf-49aa-bd6b-cf9d2494628d None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:05.933448 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba0f937-d3bf-49aa-bd6b-cf9d2494628d None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:05.933557 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba0f937-d3bf-49aa-bd6b-cf9d2494628d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:05.953033 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-eba0f937-d3bf-49aa-bd6b-cf9d2494628d None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:30:05.954352 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-eba0f937-d3bf-49aa-bd6b-cf9d2494628d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:30:06.013816 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1242/2469] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:30:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:30:06.022093 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e960c1-d397-4b23-94f0-34988f86f4b4 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:06.022405 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e960c1-d397-4b23-94f0-34988f86f4b4 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:06.022684 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e960c1-d397-4b23-94f0-34988f86f4b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:06.042459 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-f3e960c1-d397-4b23-94f0-34988f86f4b4 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:30:06.043665 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-f3e960c1-d397-4b23-94f0-34988f86f4b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:30:06.051479 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1228/2470] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:30:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:30:06.055731 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba0f937-d3bf-49aa-bd6b-cf9d2494628d None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:30:06.065498 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba0f937-d3bf-49aa-bd6b-cf9d2494628d None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:30:06.070019 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a34d7fb-daa6-4fbf-a8f0-3da1bd4d2a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:30:06.070641 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a34d7fb-daa6-4fbf-a8f0-3da1bd4d2a0c admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:06.070922 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a34d7fb-daa6-4fbf-a8f0-3da1bd4d2a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:06.071196 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a34d7fb-daa6-4fbf-a8f0-3da1bd4d2a0c admin admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:06.071841 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a34d7fb-daa6-4fbf-a8f0-3da1bd4d2a0c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:30:06.072142 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a34d7fb-daa6-4fbf-a8f0-3da1bd4d2a0c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:30:06.073766 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a34d7fb-daa6-4fbf-a8f0-3da1bd4d2a0c admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:30:06.085481 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1243/2471] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Aug 25 03:30:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:30:06.089632 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e960c1-d397-4b23-94f0-34988f86f4b4 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:30:06.099359 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e960c1-d397-4b23-94f0-34988f86f4b4 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:30:06.104020 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6197fb-e4ab-475d-9021-d2a55a49258c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:30:06.104618 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6197fb-e4ab-475d-9021-d2a55a49258c admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:06.104903 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6197fb-e4ab-475d-9021-d2a55a49258c admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:06.105176 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6197fb-e4ab-475d-9021-d2a55a49258c admin admin] PATH_INFO: `/v3/projects` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:06.105789 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6197fb-e4ab-475d-9021-d2a55a49258c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:30:06.107710 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6197fb-e4ab-475d-9021-d2a55a49258c admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:30:06.235354 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1229/2472] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:30:06 2022] POST /identity/v3/projects => generated 371 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:30:06.239305 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a34d7fb-daa6-4fbf-a8f0-3da1bd4d2a0c admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:30:06.249248 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a34d7fb-daa6-4fbf-a8f0-3da1bd4d2a0c admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:30:06.306047 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0691df-a812-43eb-951a-90b1f493af71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:30:06.306586 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0691df-a812-43eb-951a-90b1f493af71 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:06.306863 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0691df-a812-43eb-951a-90b1f493af71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:06.307158 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0691df-a812-43eb-951a-90b1f493af71 admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:06.307769 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0691df-a812-43eb-951a-90b1f493af71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:30:06.309710 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0691df-a812-43eb-951a-90b1f493af71 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:30:06.317686 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:30:06.388639 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1244/2473] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:30:06 2022] POST /identity/v3/users => generated 286 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:30:06.393165 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6197fb-e4ab-475d-9021-d2a55a49258c admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:30:06.406163 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6197fb-e4ab-475d-9021-d2a55a49258c admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:30:06.412558 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be339cc-7db0-45ed-979e-2a706fee8c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:30:06.413231 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be339cc-7db0-45ed-979e-2a706fee8c91 admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:06.413659 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be339cc-7db0-45ed-979e-2a706fee8c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:06.414049 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be339cc-7db0-45ed-979e-2a706fee8c91 admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:06.414888 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be339cc-7db0-45ed-979e-2a706fee8c91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:30:06.416849 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be339cc-7db0-45ed-979e-2a706fee8c91 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:30:06.430988 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1230/2474] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:30:06 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:30:06.436150 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0691df-a812-43eb-951a-90b1f493af71 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:30:06.448499 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0691df-a812-43eb-951a-90b1f493af71 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:30:06.453326 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bfa949-2b10-4d5c-86ec-e35879be57ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:30:06.453939 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bfa949-2b10-4d5c-86ec-e35879be57ea admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:06.454261 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bfa949-2b10-4d5c-86ec-e35879be57ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:06.454667 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bfa949-2b10-4d5c-86ec-e35879be57ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:06.455311 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bfa949-2b10-4d5c-86ec-e35879be57ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:30:06.457051 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bfa949-2b10-4d5c-86ec-e35879be57ea admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:30:06.468307 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1245/2475] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:30:06 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:30:06.472887 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be339cc-7db0-45ed-979e-2a706fee8c91 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:30:06.483245 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be339cc-7db0-45ed-979e-2a706fee8c91 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:30:06.487983 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f4be2f-63a8-4283-a798-4c3c4e926074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:30:06.494814 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f4be2f-63a8-4283-a798-4c3c4e926074 admin admin] REQUEST_METHOD: `PUT` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:06.494814 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f4be2f-63a8-4283-a798-4c3c4e926074 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:06.494814 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f4be2f-63a8-4283-a798-4c3c4e926074 admin admin] PATH_INFO: `/v3/projects/d0067a677f744be08ca3a14794e688ea/users/95057bde266143a8923bcf0f9a3f81e8/roles/3255f77192524a16bc405173106871fc` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:06.503481 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f4be2f-63a8-4283-a798-4c3c4e926074 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0067a677f744be08ca3a14794e688ea, user_id=95057bde266143a8923bcf0f9a3f81e8, role_id=3255f77192524a16bc405173106871fc)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:30:06.506217 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f4be2f-63a8-4283-a798-4c3c4e926074 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:30:06.595305 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1231/2476] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:30:06 2022] PUT /identity/v3/projects/d0067a677f744be08ca3a14794e688ea/users/95057bde266143a8923bcf0f9a3f81e8/roles/3255f77192524a16bc405173106871fc => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:30:06.605568 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a916c1b-f3e8-4f96-b952-27a90ed71661 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:06.605925 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a916c1b-f3e8-4f96-b952-27a90ed71661 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:06.606272 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a916c1b-f3e8-4f96-b952-27a90ed71661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:06.635352 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-1a916c1b-f3e8-4f96-b952-27a90ed71661 None None] MFA Rules not processed for user `95057bde266143a8923bcf0f9a3f81e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:30:06.636984 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-1a916c1b-f3e8-4f96-b952-27a90ed71661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:30:06.703759 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1246/2477] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:30:06 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:30:06.714650 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f4be2f-63a8-4283-a798-4c3c4e926074 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:30:06.716968 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-c9f4be2f-63a8-4283-a798-4c3c4e926074 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:30:06.791153 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f4be2f-63a8-4283-a798-4c3c4e926074 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:30:06.796757 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e4f8f5-295e-4e26-9543-e219e08ae84d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cd171bf4df94f8fab397e42835feeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:30:06.797395 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e4f8f5-295e-4e26-9543-e219e08ae84d service nova] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:06.797675 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e4f8f5-295e-4e26-9543-e219e08ae84d service nova] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:06.797950 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e4f8f5-295e-4e26-9543-e219e08ae84d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:06.805565 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e4f8f5-295e-4e26-9543-e219e08ae84d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:30:06.807180 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e4f8f5-295e-4e26-9543-e219e08ae84d service nova] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:30:06.816217 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1232/2478] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:30:06 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:30:08.643500 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a916c1b-f3e8-4f96-b952-27a90ed71661 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:30:08.656701 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a916c1b-f3e8-4f96-b952-27a90ed71661 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:30:08.662142 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca92e61-ce2a-45ab-88aa-e1d593008d50 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cd171bf4df94f8fab397e42835feeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:30:08.662787 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca92e61-ce2a-45ab-88aa-e1d593008d50 service nova] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:08.663079 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca92e61-ce2a-45ab-88aa-e1d593008d50 service nova] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:08.663334 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca92e61-ce2a-45ab-88aa-e1d593008d50 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:08.665067 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-9ca92e61-ce2a-45ab-88aa-e1d593008d50 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:30:08.677517 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca92e61-ce2a-45ab-88aa-e1d593008d50 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:30:08.679219 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca92e61-ce2a-45ab-88aa-e1d593008d50 service nova] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:30:08.690744 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1247/2479] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:30:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:30:10.033246 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e4f8f5-295e-4e26-9543-e219e08ae84d service nova] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:30:10.041891 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:30:10.102502 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e4f8f5-295e-4e26-9543-e219e08ae84d service nova] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:30:10.107776 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d863c764-a6ed-46fc-9118-f4abfa11be33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:30:10.112684 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d863c764-a6ed-46fc-9118-f4abfa11be33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:10.112684 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d863c764-a6ed-46fc-9118-f4abfa11be33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:10.112684 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d863c764-a6ed-46fc-9118-f4abfa11be33 admin admin] PATH_INFO: `/v3/users/95057bde266143a8923bcf0f9a3f81e8` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:10.112684 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d863c764-a6ed-46fc-9118-f4abfa11be33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95057bde266143a8923bcf0f9a3f81e8)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:30:10.112684 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d863c764-a6ed-46fc-9118-f4abfa11be33 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:30:10.330543 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-d863c764-a6ed-46fc-9118-f4abfa11be33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95057bde266143a8923bcf0f9a3f81e8'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:30:10.427595 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-d863c764-a6ed-46fc-9118-f4abfa11be33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95057bde266143a8923bcf0f9a3f81e8'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:30:10.428158 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-d863c764-a6ed-46fc-9118-f4abfa11be33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95057bde266143a8923bcf0f9a3f81e8'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:30:10.428884 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-d863c764-a6ed-46fc-9118-f4abfa11be33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95057bde266143a8923bcf0f9a3f81e8'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:30:10.443562 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-d863c764-a6ed-46fc-9118-f4abfa11be33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95057bde266143a8923bcf0f9a3f81e8'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:30:10.456412 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1233/2480] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:30:10 2022] DELETE /identity/v3/users/95057bde266143a8923bcf0f9a3f81e8 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:30:10.461117 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd71d6d8-b945-437d-a918-291628195d1d None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:10.461593 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd71d6d8-b945-437d-a918-291628195d1d None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:10.461692 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd71d6d8-b945-437d-a918-291628195d1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:10.481047 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-dd71d6d8-b945-437d-a918-291628195d1d None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:30:10.482955 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-dd71d6d8-b945-437d-a918-291628195d1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:30:10.547211 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1248/2481] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:30:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:30:10.556870 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d863c764-a6ed-46fc-9118-f4abfa11be33 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:30:10.563177 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-d863c764-a6ed-46fc-9118-f4abfa11be33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:30:10.615577 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d863c764-a6ed-46fc-9118-f4abfa11be33 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:30:10.619156 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4171b3-0310-49a9-8408-3bd784ee93f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:30:10.619836 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4171b3-0310-49a9-8408-3bd784ee93f5 service neutron] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:10.620159 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4171b3-0310-49a9-8408-3bd784ee93f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:10.620434 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4171b3-0310-49a9-8408-3bd784ee93f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:10.629518 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4171b3-0310-49a9-8408-3bd784ee93f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:30:10.631004 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4171b3-0310-49a9-8408-3bd784ee93f5 service neutron] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:30:10.643463 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1234/2482] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:30:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:30:11.046506 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd71d6d8-b945-437d-a918-291628195d1d None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:30:11.047955 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-dd71d6d8-b945-437d-a918-291628195d1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:30:11.064959 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd71d6d8-b945-437d-a918-291628195d1d None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:30:11.071173 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b0332e-db35-451f-addd-a8c68ce69790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:30:11.076443 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b0332e-db35-451f-addd-a8c68ce69790 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:11.076443 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b0332e-db35-451f-addd-a8c68ce69790 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:11.076443 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b0332e-db35-451f-addd-a8c68ce69790 admin admin] PATH_INFO: `/v3/projects/d0067a677f744be08ca3a14794e688ea` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:11.076443 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b0332e-db35-451f-addd-a8c68ce69790 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0067a677f744be08ca3a14794e688ea)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:30:11.076443 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b0332e-db35-451f-addd-a8c68ce69790 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:30:11.171904 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-e3b0332e-db35-451f-addd-a8c68ce69790 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0067a677f744be08ca3a14794e688ea', 'request_id': 'req-e3b0332e-db35-451f-addd-a8c68ce69790'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:30:11.173623 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:INSERT=3,keystone:SELECT=93,keystone:DELETE=4 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:30:11.262096 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-e3b0332e-db35-451f-addd-a8c68ce69790 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0067a677f744be08ca3a14794e688ea', 'request_id': 'req-e3b0332e-db35-451f-addd-a8c68ce69790'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:30:11.269333 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-e3b0332e-db35-451f-addd-a8c68ce69790 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0067a677f744be08ca3a14794e688ea', 'request_id': 'req-e3b0332e-db35-451f-addd-a8c68ce69790'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:30:11.280903 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-e3b0332e-db35-451f-addd-a8c68ce69790 admin admin] The token cache is being invalidate because project d0067a677f744be08ca3a14794e688ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:30:11.281310 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-e3b0332e-db35-451f-addd-a8c68ce69790 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:30:11.282309 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-e3b0332e-db35-451f-addd-a8c68ce69790 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:30:11.285667 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1249/2483] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:30:11 2022] DELETE /identity/v3/projects/d0067a677f744be08ca3a14794e688ea => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:30:11.290493 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3316fb2-dcf4-4ccc-a6d1-2268ace601e1 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:11.290900 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3316fb2-dcf4-4ccc-a6d1-2268ace601e1 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:11.291159 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3316fb2-dcf4-4ccc-a6d1-2268ace601e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:11.312266 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-b3316fb2-dcf4-4ccc-a6d1-2268ace601e1 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:30:11.313596 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-b3316fb2-dcf4-4ccc-a6d1-2268ace601e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:30:11.350779 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:30:11.391901 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1235/2484] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:30:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:30:11.400080 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67786307-66f5-4bb2-ab3c-1072b3414a7b None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:11.400389 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67786307-66f5-4bb2-ab3c-1072b3414a7b None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:11.400674 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67786307-66f5-4bb2-ab3c-1072b3414a7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:11.423295 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-67786307-66f5-4bb2-ab3c-1072b3414a7b None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:30:11.424540 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-67786307-66f5-4bb2-ab3c-1072b3414a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:30:11.432763 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1250/2485] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:30:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:30:11.438426 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3316fb2-dcf4-4ccc-a6d1-2268ace601e1 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:30:11.448845 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3316fb2-dcf4-4ccc-a6d1-2268ace601e1 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:30:11.453769 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4da5be-2d5f-4f31-af8f-91f1e7851c5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:30:11.454503 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4da5be-2d5f-4f31-af8f-91f1e7851c5c admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:11.454850 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4da5be-2d5f-4f31-af8f-91f1e7851c5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:11.455217 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4da5be-2d5f-4f31-af8f-91f1e7851c5c admin admin] PATH_INFO: `/v3/domains` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:11.455957 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4da5be-2d5f-4f31-af8f-91f1e7851c5c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:30:11.456331 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4da5be-2d5f-4f31-af8f-91f1e7851c5c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:30:11.458023 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4da5be-2d5f-4f31-af8f-91f1e7851c5c admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:30:11.470231 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1236/2486] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Aug 25 03:30:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:30:11.474837 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67786307-66f5-4bb2-ab3c-1072b3414a7b None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:30:11.485037 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67786307-66f5-4bb2-ab3c-1072b3414a7b None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:30:11.487472 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4da5be-2d5f-4f31-af8f-91f1e7851c5c admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:30:11.489229 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-ee4da5be-2d5f-4f31-af8f-91f1e7851c5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:30:11.490320 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98167e6-b65d-4e10-b886-b6c9ce1015d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:30:11.490978 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98167e6-b65d-4e10-b886-b6c9ce1015d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:11.491258 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98167e6-b65d-4e10-b886-b6c9ce1015d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:11.491530 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98167e6-b65d-4e10-b886-b6c9ce1015d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:11.492158 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98167e6-b65d-4e10-b886-b6c9ce1015d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:30:11.494313 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98167e6-b65d-4e10-b886-b6c9ce1015d8 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:30:11.552009 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4da5be-2d5f-4f31-af8f-91f1e7851c5c admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:30:11.555703 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead469e7-24df-42f9-91e8-60ddb34a6881 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:30:11.556415 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead469e7-24df-42f9-91e8-60ddb34a6881 service neutron] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:11.556726 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead469e7-24df-42f9-91e8-60ddb34a6881 service neutron] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:11.557003 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead469e7-24df-42f9-91e8-60ddb34a6881 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:11.558766 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-ead469e7-24df-42f9-91e8-60ddb34a6881 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:30:11.569277 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead469e7-24df-42f9-91e8-60ddb34a6881 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:30:11.573295 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead469e7-24df-42f9-91e8-60ddb34a6881 service neutron] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:30:11.581794 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1237/2487] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:30:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 25 03:30:11.605826 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1251/2488] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:30:11 2022] POST /identity/v3/projects => generated 365 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:30:11.606010 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead469e7-24df-42f9-91e8-60ddb34a6881 service neutron] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:30:11.616956 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead469e7-24df-42f9-91e8-60ddb34a6881 service neutron] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:30:11.672665 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b7abb9-1390-4426-83e5-e27401d382b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:30:11.673273 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b7abb9-1390-4426-83e5-e27401d382b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:11.673555 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b7abb9-1390-4426-83e5-e27401d382b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:11.673831 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b7abb9-1390-4426-83e5-e27401d382b0 admin admin] PATH_INFO: `/v3/users` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:11.674487 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b7abb9-1390-4426-83e5-e27401d382b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:30:11.676428 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b7abb9-1390-4426-83e5-e27401d382b0 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:30:11.742295 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1238/2489] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:30:11 2022] POST /identity/v3/users => generated 283 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:30:11.746534 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98167e6-b65d-4e10-b886-b6c9ce1015d8 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:30:11.756886 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98167e6-b65d-4e10-b886-b6c9ce1015d8 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:30:11.761848 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e563e28-5e4c-4d1d-a89a-7d96118ca8d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:30:11.762512 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e563e28-5e4c-4d1d-a89a-7d96118ca8d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:11.762801 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e563e28-5e4c-4d1d-a89a-7d96118ca8d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:11.763114 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e563e28-5e4c-4d1d-a89a-7d96118ca8d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:11.763882 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e563e28-5e4c-4d1d-a89a-7d96118ca8d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:30:11.765483 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e563e28-5e4c-4d1d-a89a-7d96118ca8d9 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:30:11.776016 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1252/2490] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:30:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:30:11.780864 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b7abb9-1390-4426-83e5-e27401d382b0 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:30:11.790989 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b7abb9-1390-4426-83e5-e27401d382b0 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:30:11.796260 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542c272f-d354-477e-b000-6d70925185a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:30:11.796854 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542c272f-d354-477e-b000-6d70925185a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:11.797134 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542c272f-d354-477e-b000-6d70925185a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:11.797499 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542c272f-d354-477e-b000-6d70925185a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:11.798165 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542c272f-d354-477e-b000-6d70925185a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:30:11.799868 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542c272f-d354-477e-b000-6d70925185a9 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:30:11.813213 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1239/2491] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:30:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:30:11.814254 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e563e28-5e4c-4d1d-a89a-7d96118ca8d9 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:30:11.825598 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e563e28-5e4c-4d1d-a89a-7d96118ca8d9 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:30:11.830881 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1490dd-f850-4c36-b66c-2f42f4f1b4d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:30:11.831557 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1490dd-f850-4c36-b66c-2f42f4f1b4d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:11.832330 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1490dd-f850-4c36-b66c-2f42f4f1b4d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:11.832330 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1490dd-f850-4c36-b66c-2f42f4f1b4d2 admin admin] PATH_INFO: `/v3/projects/eb226cd9b0a9401881e8f2e7ea51ae48/users/cff37966ce544473a898fe8a47cec933/roles/3255f77192524a16bc405173106871fc` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:11.849636 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1490dd-f850-4c36-b66c-2f42f4f1b4d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb226cd9b0a9401881e8f2e7ea51ae48, user_id=cff37966ce544473a898fe8a47cec933, role_id=3255f77192524a16bc405173106871fc)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:30:11.852457 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1490dd-f850-4c36-b66c-2f42f4f1b4d2 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:30:11.933241 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c461404d-68e1-4449-add5-7f09d6fb9c7f None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:11.933558 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c461404d-68e1-4449-add5-7f09d6fb9c7f None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:11.933988 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c461404d-68e1-4449-add5-7f09d6fb9c7f None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:11.935805 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1240/2492] 10.222.0.58 () {54 vars in 921 bytes} [Thu Aug 25 03:30:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:30:11.938541 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1253/2493] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:30:11 2022] PUT /identity/v3/projects/eb226cd9b0a9401881e8f2e7ea51ae48/users/cff37966ce544473a898fe8a47cec933/roles/3255f77192524a16bc405173106871fc => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:30:11.942571 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a8c822-4988-4828-805e-cf0945711261 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:11.942994 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a8c822-4988-4828-805e-cf0945711261 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:11.943304 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a8c822-4988-4828-805e-cf0945711261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:11.950190 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbd91ed-3170-44af-9c47-56232baf1a5b None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:11.950655 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbd91ed-3170-44af-9c47-56232baf1a5b None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:11.950939 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbd91ed-3170-44af-9c47-56232baf1a5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:11.982043 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-ecbd91ed-3170-44af-9c47-56232baf1a5b None None] MFA Rules not processed for user `cff37966ce544473a898fe8a47cec933`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:30:11.983820 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-ecbd91ed-3170-44af-9c47-56232baf1a5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:30:11.990383 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-e2a8c822-4988-4828-805e-cf0945711261 None None] MFA Rules not processed for user `0cd171bf4df94f8fab397e42835feeab`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:30:11.991786 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-e2a8c822-4988-4828-805e-cf0945711261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:30:12.014395 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:30:12.058524 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1254/2494] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:30:11 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:30:12.063364 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbd91ed-3170-44af-9c47-56232baf1a5b None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:30:12.066007 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1241/2495] 10.222.0.58 () {58 vars in 1018 bytes} [Thu Aug 25 03:30:11 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:30:12.078059 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a8c822-4988-4828-805e-cf0945711261 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:30:12.080036 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-e2a8c822-4988-4828-805e-cf0945711261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:30:12.125873 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbd91ed-3170-44af-9c47-56232baf1a5b None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:30:12.130842 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c221b59-b838-4cf9-bf76-60986bc57103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:30:12.131679 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c221b59-b838-4cf9-bf76-60986bc57103 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:12.132151 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c221b59-b838-4cf9-bf76-60986bc57103 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:12.132438 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c221b59-b838-4cf9-bf76-60986bc57103 admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:12.133449 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c221b59-b838-4cf9-bf76-60986bc57103 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:30:12.135208 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c221b59-b838-4cf9-bf76-60986bc57103 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:30:12.138630 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a8c822-4988-4828-805e-cf0945711261 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:30:12.142733 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3959b2-433f-42d4-a5cb-442da1167eaf service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f89089b390c4b6599e54000c0fa082a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:30:12.143430 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3959b2-433f-42d4-a5cb-442da1167eaf service glance] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:12.143833 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3959b2-433f-42d4-a5cb-442da1167eaf service glance] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:12.143964 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3959b2-433f-42d4-a5cb-442da1167eaf service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:12.146532 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:INSERT=3,keystone:DELETE=3,keystone:SELECT=94 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:30:12.155238 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3959b2-433f-42d4-a5cb-442da1167eaf service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:30:12.156676 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3959b2-433f-42d4-a5cb-442da1167eaf service glance] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:30:12.167369 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1242/2496] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:30:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 25 03:30:12.249604 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1255/2497] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:30:12 2022] POST /identity/v3/projects => generated 361 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:30:12.250795 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3959b2-433f-42d4-a5cb-442da1167eaf service glance] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:30:12.261545 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3959b2-433f-42d4-a5cb-442da1167eaf service glance] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:30:12.323500 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2db7141-2e4d-482a-aa5f-508135cfc32b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:30:12.324273 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2db7141-2e4d-482a-aa5f-508135cfc32b admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:12.324437 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2db7141-2e4d-482a-aa5f-508135cfc32b admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:12.324639 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2db7141-2e4d-482a-aa5f-508135cfc32b admin admin] PATH_INFO: `/v3/users` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:12.325273 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2db7141-2e4d-482a-aa5f-508135cfc32b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:30:12.328327 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2db7141-2e4d-482a-aa5f-508135cfc32b admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:30:12.372206 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7509afde-5e54-4654-9874-278960b30f07 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:12.372531 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7509afde-5e54-4654-9874-278960b30f07 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:12.372827 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7509afde-5e54-4654-9874-278960b30f07 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:12.374227 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1256/2498] 10.222.0.58 () {54 vars in 921 bytes} [Thu Aug 25 03:30:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 25 03:30:12.380551 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef468e46-7926-4993-9341-27f89db38adb None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:12.380830 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef468e46-7926-4993-9341-27f89db38adb None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:12.381106 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef468e46-7926-4993-9341-27f89db38adb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:12.394446 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1243/2499] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:30:12 2022] POST /identity/v3/users => generated 287 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:30:12.398579 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2db7141-2e4d-482a-aa5f-508135cfc32b admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:30:12.409162 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2db7141-2e4d-482a-aa5f-508135cfc32b admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:30:12.413871 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb83b93-1441-4ba1-96c2-591e33694c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:30:12.414487 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb83b93-1441-4ba1-96c2-591e33694c82 admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:12.414808 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb83b93-1441-4ba1-96c2-591e33694c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:12.415079 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb83b93-1441-4ba1-96c2-591e33694c82 admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:12.415719 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb83b93-1441-4ba1-96c2-591e33694c82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:30:12.417336 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb83b93-1441-4ba1-96c2-591e33694c82 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:30:12.420463 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-ef468e46-7926-4993-9341-27f89db38adb None None] MFA Rules not processed for user `a0a1a7f091774842907fd08de15e8058`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:30:12.421675 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-ef468e46-7926-4993-9341-27f89db38adb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:30:12.427421 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1244/2500] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:30:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 25 03:30:12.431592 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb83b93-1441-4ba1-96c2-591e33694c82 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:30:12.442347 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb83b93-1441-4ba1-96c2-591e33694c82 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:30:12.447104 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362a40ac-10c3-472b-a172-762d0b6d54dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:30:12.447747 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362a40ac-10c3-472b-a172-762d0b6d54dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:12.448035 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362a40ac-10c3-472b-a172-762d0b6d54dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:12.448317 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362a40ac-10c3-472b-a172-762d0b6d54dd admin admin] PATH_INFO: `/v3/projects/54a56eefb1cd45a196cdf268c09a0f91/users/78f321b26707482e878a4b9078fa9e1e/roles/62f19ea272b447ccaab1576cd9bf4fd9` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:12.464905 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362a40ac-10c3-472b-a172-762d0b6d54dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=54a56eefb1cd45a196cdf268c09a0f91, user_id=78f321b26707482e878a4b9078fa9e1e, role_id=62f19ea272b447ccaab1576cd9bf4fd9)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:30:12.468254 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362a40ac-10c3-472b-a172-762d0b6d54dd admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:30:12.475412 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1257/2501] 10.222.0.58 () {58 vars in 1018 bytes} [Thu Aug 25 03:30:12 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:30:12.487232 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef468e46-7926-4993-9341-27f89db38adb None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:30:12.488599 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-ef468e46-7926-4993-9341-27f89db38adb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:30:12.499178 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef468e46-7926-4993-9341-27f89db38adb None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:30:12.502830 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1029f5c-a1c2-4920-85c3-59dfd8440510 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:30:12.503437 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1029f5c-a1c2-4920-85c3-59dfd8440510 service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:12.503696 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1029f5c-a1c2-4920-85c3-59dfd8440510 service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:12.503966 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1029f5c-a1c2-4920-85c3-59dfd8440510 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:12.510819 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1029f5c-a1c2-4920-85c3-59dfd8440510 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:30:12.512307 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1029f5c-a1c2-4920-85c3-59dfd8440510 service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:30:12.520632 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1258/2502] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:30:12 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 25 03:30:12.542011 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1245/2503] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:30:12 2022] PUT /identity/v3/projects/54a56eefb1cd45a196cdf268c09a0f91/users/78f321b26707482e878a4b9078fa9e1e/roles/62f19ea272b447ccaab1576cd9bf4fd9 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:30:12.545964 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1029f5c-a1c2-4920-85c3-59dfd8440510 service neutron] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:30:12.557274 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1029f5c-a1c2-4920-85c3-59dfd8440510 service neutron] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:30:12.619512 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172dce17-ef35-4424-aa21-5bcb23f5234d None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:12.619790 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172dce17-ef35-4424-aa21-5bcb23f5234d None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:12.620062 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172dce17-ef35-4424-aa21-5bcb23f5234d None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:12.621394 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1246/2504] 10.222.0.58 () {54 vars in 921 bytes} [Thu Aug 25 03:30:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:30:12.626558 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07edae4b-97fc-4133-a3dd-66bcc9f4daa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:30:12.627184 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07edae4b-97fc-4133-a3dd-66bcc9f4daa6 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:12.627546 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07edae4b-97fc-4133-a3dd-66bcc9f4daa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:12.627823 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07edae4b-97fc-4133-a3dd-66bcc9f4daa6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:12.628466 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07edae4b-97fc-4133-a3dd-66bcc9f4daa6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:30:12.628962 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778b0f44-8045-46d1-a9e5-d4855f5537ba None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:12.629336 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778b0f44-8045-46d1-a9e5-d4855f5537ba None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:12.629700 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778b0f44-8045-46d1-a9e5-d4855f5537ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:12.634739 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07edae4b-97fc-4133-a3dd-66bcc9f4daa6 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:30:12.647716 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1259/2505] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:30:12 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:30:12.651936 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-778b0f44-8045-46d1-a9e5-d4855f5537ba None None] MFA Rules not processed for user `0cd171bf4df94f8fab397e42835feeab`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:30:12.652079 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07edae4b-97fc-4133-a3dd-66bcc9f4daa6 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:30:12.653331 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-778b0f44-8045-46d1-a9e5-d4855f5537ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:30:12.664523 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07edae4b-97fc-4133-a3dd-66bcc9f4daa6 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:30:12.664670 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:30:12.667889 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b3e11c-8be4-44d8-9c46-fa58673d45a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:30:12.668594 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b3e11c-8be4-44d8-9c46-fa58673d45a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:12.668880 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b3e11c-8be4-44d8-9c46-fa58673d45a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:12.669159 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b3e11c-8be4-44d8-9c46-fa58673d45a7 admin admin] PATH_INFO: `/v3/projects/54a56eefb1cd45a196cdf268c09a0f91/users/78f321b26707482e878a4b9078fa9e1e/roles/62f19ea272b447ccaab1576cd9bf4fd9` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:12.671195 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b3e11c-8be4-44d8-9c46-fa58673d45a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54a56eefb1cd45a196cdf268c09a0f91, user_id=78f321b26707482e878a4b9078fa9e1e, role_id=62f19ea272b447ccaab1576cd9bf4fd9)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:30:12.674015 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b3e11c-8be4-44d8-9c46-fa58673d45a7 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:30:12.686666 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1260/2506] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:30:12 2022] PUT /identity/v3/projects/54a56eefb1cd45a196cdf268c09a0f91/users/78f321b26707482e878a4b9078fa9e1e/roles/62f19ea272b447ccaab1576cd9bf4fd9 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:30:12.697088 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff208c0-20f7-4249-a57f-13cb122c07a9 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:12.697380 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff208c0-20f7-4249-a57f-13cb122c07a9 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:12.697724 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff208c0-20f7-4249-a57f-13cb122c07a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:12.725982 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-4ff208c0-20f7-4249-a57f-13cb122c07a9 None None] MFA Rules not processed for user `78f321b26707482e878a4b9078fa9e1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:30:12.727261 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-4ff208c0-20f7-4249-a57f-13cb122c07a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:30:12.730429 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1247/2507] 10.222.0.58 () {58 vars in 1018 bytes} [Thu Aug 25 03:30:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:30:12.741070 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778b0f44-8045-46d1-a9e5-d4855f5537ba None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:30:12.772558 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:30:12.796154 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778b0f44-8045-46d1-a9e5-d4855f5537ba None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:30:12.798694 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1261/2508] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:30:12 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:30:12.811512 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff208c0-20f7-4249-a57f-13cb122c07a9 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:30:12.813017 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-4ff208c0-20f7-4249-a57f-13cb122c07a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:30:12.850366 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d799fc7b-94a3-401f-a6b2-63f3d7e0a8a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:30:12.850984 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d799fc7b-94a3-401f-a6b2-63f3d7e0a8a7 service neutron] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:12.851260 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d799fc7b-94a3-401f-a6b2-63f3d7e0a8a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:12.851534 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d799fc7b-94a3-401f-a6b2-63f3d7e0a8a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:12.880714 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff208c0-20f7-4249-a57f-13cb122c07a9 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:30:12.884460 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c7807e-d660-4cfc-9b80-c65e46aa5e16 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60193ba9e1c14df3bc9eebcc6f28f2d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:30:12.885433 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c7807e-d660-4cfc-9b80-c65e46aa5e16 service cinder] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:12.885714 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c7807e-d660-4cfc-9b80-c65e46aa5e16 service cinder] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:12.886025 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c7807e-d660-4cfc-9b80-c65e46aa5e16 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:12.897531 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c7807e-d660-4cfc-9b80-c65e46aa5e16 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:30:12.899002 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c7807e-d660-4cfc-9b80-c65e46aa5e16 service cinder] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:30:12.908732 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1262/2509] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:30:12 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 25 03:30:12.922891 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d799fc7b-94a3-401f-a6b2-63f3d7e0a8a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:30:12.924337 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d799fc7b-94a3-401f-a6b2-63f3d7e0a8a7 service neutron] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:30:12.936074 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1248/2510] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:30:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 25 03:30:13.129625 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c7807e-d660-4cfc-9b80-c65e46aa5e16 service cinder] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:30:13.139616 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c7807e-d660-4cfc-9b80-c65e46aa5e16 service cinder] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:30:13.143681 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c06e47-b27d-4de7-8c36-1257bd82e60a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60193ba9e1c14df3bc9eebcc6f28f2d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:30:13.144564 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c06e47-b27d-4de7-8c36-1257bd82e60a service cinder] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:13.145314 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c06e47-b27d-4de7-8c36-1257bd82e60a service cinder] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:13.145314 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c06e47-b27d-4de7-8c36-1257bd82e60a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:13.208535 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c06e47-b27d-4de7-8c36-1257bd82e60a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:30:13.210062 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c06e47-b27d-4de7-8c36-1257bd82e60a service cinder] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:30:13.222244 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1263/2511] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:30:13 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:30:16.857727 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d413ab-aff8-4f7d-b196-8982f86d749c None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:16.857727 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d413ab-aff8-4f7d-b196-8982f86d749c None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:16.858346 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d413ab-aff8-4f7d-b196-8982f86d749c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:16.865165 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=100 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:30:16.877573 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-b3d413ab-aff8-4f7d-b196-8982f86d749c None None] MFA Rules not processed for user `712c35b765b64919aa3eaaf02892590d`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:30:16.878800 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-b3d413ab-aff8-4f7d-b196-8982f86d749c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:30:16.933262 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1249/2512] 10.222.0.58 () {58 vars in 1009 bytes} [Thu Aug 25 03:30:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:30:16.945452 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c06e47-b27d-4de7-8c36-1257bd82e60a service cinder] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:30:16.948044 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-c5c06e47-b27d-4de7-8c36-1257bd82e60a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:30:17.018985 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c06e47-b27d-4de7-8c36-1257bd82e60a service cinder] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:30:17.024250 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c509a6dd-37ce-4710-b7f8-d35ea780543a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5047e1a7cd2d416d90b2d8c5a6596522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:30:17.024855 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c509a6dd-37ce-4710-b7f8-d35ea780543a service swift] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:17.025128 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c509a6dd-37ce-4710-b7f8-d35ea780543a service swift] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:17.025399 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c509a6dd-37ce-4710-b7f8-d35ea780543a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:17.034220 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c509a6dd-37ce-4710-b7f8-d35ea780543a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:30:17.035866 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c509a6dd-37ce-4710-b7f8-d35ea780543a service swift] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:30:17.044092 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1264/2513] 10.222.0.58 () {60 vars in 1384 bytes} [Thu Aug 25 03:30:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 25 03:30:26.931707 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=24 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:30:27.040787 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=95 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:30:32.733728 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d413ab-aff8-4f7d-b196-8982f86d749c None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:30:32.799464 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d413ab-aff8-4f7d-b196-8982f86d749c None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:30:32.804078 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483b4799-1c24-4384-8245-c9f230fca0fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:30:32.804723 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483b4799-1c24-4384-8245-c9f230fca0fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:32.805001 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483b4799-1c24-4384-8245-c9f230fca0fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:32.805277 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483b4799-1c24-4384-8245-c9f230fca0fd admin admin] PATH_INFO: `/v3/users/cff37966ce544473a898fe8a47cec933` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:32.806456 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483b4799-1c24-4384-8245-c9f230fca0fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=cff37966ce544473a898fe8a47cec933)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:30:32.808396 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483b4799-1c24-4384-8245-c9f230fca0fd admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:30:32.979083 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-483b4799-1c24-4384-8245-c9f230fca0fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cff37966ce544473a898fe8a47cec933'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:30:33.056073 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-483b4799-1c24-4384-8245-c9f230fca0fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cff37966ce544473a898fe8a47cec933'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:30:33.056803 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-483b4799-1c24-4384-8245-c9f230fca0fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cff37966ce544473a898fe8a47cec933'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:30:33.057437 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-483b4799-1c24-4384-8245-c9f230fca0fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cff37966ce544473a898fe8a47cec933'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:30:33.071216 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-483b4799-1c24-4384-8245-c9f230fca0fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cff37966ce544473a898fe8a47cec933'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:30:33.084039 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1250/2514] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:30:32 2022] DELETE /identity/v3/users/cff37966ce544473a898fe8a47cec933 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:30:33.088629 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d23b08-17aa-4590-b916-1aec89d7d77e None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:33.093240 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d23b08-17aa-4590-b916-1aec89d7d77e None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:33.093240 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d23b08-17aa-4590-b916-1aec89d7d77e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:33.108708 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-c4d23b08-17aa-4590-b916-1aec89d7d77e None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:30:33.110413 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-c4d23b08-17aa-4590-b916-1aec89d7d77e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:30:33.172784 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1265/2515] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:30:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:30:33.182632 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483b4799-1c24-4384-8245-c9f230fca0fd admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:30:33.184037 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-483b4799-1c24-4384-8245-c9f230fca0fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:30:33.253126 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483b4799-1c24-4384-8245-c9f230fca0fd admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:30:33.256965 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e560bb3-dd69-4b00-87ca-ce21b0402e5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:30:33.257582 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e560bb3-dd69-4b00-87ca-ce21b0402e5b service neutron] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:33.257863 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e560bb3-dd69-4b00-87ca-ce21b0402e5b service neutron] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:33.258137 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e560bb3-dd69-4b00-87ca-ce21b0402e5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:33.267698 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e560bb3-dd69-4b00-87ca-ce21b0402e5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:30:33.269144 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e560bb3-dd69-4b00-87ca-ce21b0402e5b service neutron] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:30:33.282213 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1251/2516] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:30:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:30:33.731766 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d23b08-17aa-4590-b916-1aec89d7d77e None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:30:33.735474 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-c4d23b08-17aa-4590-b916-1aec89d7d77e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:30:33.746125 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d23b08-17aa-4590-b916-1aec89d7d77e None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:30:33.752961 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e17f57-dc99-468a-8fe9-52bae991073d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:30:33.753776 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e17f57-dc99-468a-8fe9-52bae991073d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:33.754082 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e17f57-dc99-468a-8fe9-52bae991073d admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:33.754399 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e17f57-dc99-468a-8fe9-52bae991073d admin admin] PATH_INFO: `/v3/projects/eb226cd9b0a9401881e8f2e7ea51ae48` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:33.755545 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e17f57-dc99-468a-8fe9-52bae991073d admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb226cd9b0a9401881e8f2e7ea51ae48)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:30:33.758329 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e17f57-dc99-468a-8fe9-52bae991073d admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:30:33.909636 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-83e17f57-dc99-468a-8fe9-52bae991073d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb226cd9b0a9401881e8f2e7ea51ae48', 'request_id': 'req-83e17f57-dc99-468a-8fe9-52bae991073d'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:30:33.971102 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-83e17f57-dc99-468a-8fe9-52bae991073d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb226cd9b0a9401881e8f2e7ea51ae48', 'request_id': 'req-83e17f57-dc99-468a-8fe9-52bae991073d'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:30:33.977430 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-83e17f57-dc99-468a-8fe9-52bae991073d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb226cd9b0a9401881e8f2e7ea51ae48', 'request_id': 'req-83e17f57-dc99-468a-8fe9-52bae991073d'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:30:33.989129 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-83e17f57-dc99-468a-8fe9-52bae991073d admin admin] The token cache is being invalidate because project eb226cd9b0a9401881e8f2e7ea51ae48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:30:33.989416 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-83e17f57-dc99-468a-8fe9-52bae991073d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:30:33.990062 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-83e17f57-dc99-468a-8fe9-52bae991073d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:30:33.991631 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1266/2517] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:30:33 2022] DELETE /identity/v3/projects/eb226cd9b0a9401881e8f2e7ea51ae48 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:30:33.995880 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e560bb3-dd69-4b00-87ca-ce21b0402e5b service neutron] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:30:34.006739 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e560bb3-dd69-4b00-87ca-ce21b0402e5b service neutron] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:30:34.027600 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:30:34.066817 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217e9750-7d6b-4f19-ae1d-b5a0782f8c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:30:34.067453 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217e9750-7d6b-4f19-ae1d-b5a0782f8c69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:34.067737 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217e9750-7d6b-4f19-ae1d-b5a0782f8c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:34.068014 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217e9750-7d6b-4f19-ae1d-b5a0782f8c69 admin admin] PATH_INFO: `/v3/users/78f321b26707482e878a4b9078fa9e1e` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:34.069155 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217e9750-7d6b-4f19-ae1d-b5a0782f8c69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=78f321b26707482e878a4b9078fa9e1e)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:30:34.071180 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217e9750-7d6b-4f19-ae1d-b5a0782f8c69 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:30:34.322218 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-217e9750-7d6b-4f19-ae1d-b5a0782f8c69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78f321b26707482e878a4b9078fa9e1e'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:30:34.364471 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-217e9750-7d6b-4f19-ae1d-b5a0782f8c69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78f321b26707482e878a4b9078fa9e1e'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:30:34.365104 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-217e9750-7d6b-4f19-ae1d-b5a0782f8c69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78f321b26707482e878a4b9078fa9e1e'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:30:34.365720 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-217e9750-7d6b-4f19-ae1d-b5a0782f8c69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78f321b26707482e878a4b9078fa9e1e'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:30:34.379985 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-217e9750-7d6b-4f19-ae1d-b5a0782f8c69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78f321b26707482e878a4b9078fa9e1e'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:30:34.393328 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1252/2518] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:30:33 2022] DELETE /identity/v3/users/78f321b26707482e878a4b9078fa9e1e => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:30:34.898037 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e17f57-dc99-468a-8fe9-52bae991073d admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:30:34.899668 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-83e17f57-dc99-468a-8fe9-52bae991073d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:30:34.920981 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e17f57-dc99-468a-8fe9-52bae991073d admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:30:34.982459 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14851681-7bda-49ca-8fa5-b6a3643b56a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:30:34.983099 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14851681-7bda-49ca-8fa5-b6a3643b56a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:34.983378 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14851681-7bda-49ca-8fa5-b6a3643b56a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:34.983652 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14851681-7bda-49ca-8fa5-b6a3643b56a8 admin admin] PATH_INFO: `/v3/projects/54a56eefb1cd45a196cdf268c09a0f91` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:34.984817 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14851681-7bda-49ca-8fa5-b6a3643b56a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=54a56eefb1cd45a196cdf268c09a0f91)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:30:34.986786 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14851681-7bda-49ca-8fa5-b6a3643b56a8 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:30:35.074466 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:30:35.091682 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-14851681-7bda-49ca-8fa5-b6a3643b56a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54a56eefb1cd45a196cdf268c09a0f91', 'request_id': 'req-14851681-7bda-49ca-8fa5-b6a3643b56a8'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:30:35.170837 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-14851681-7bda-49ca-8fa5-b6a3643b56a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54a56eefb1cd45a196cdf268c09a0f91', 'request_id': 'req-14851681-7bda-49ca-8fa5-b6a3643b56a8'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:30:35.177966 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-14851681-7bda-49ca-8fa5-b6a3643b56a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54a56eefb1cd45a196cdf268c09a0f91', 'request_id': 'req-14851681-7bda-49ca-8fa5-b6a3643b56a8'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:30:35.190877 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-14851681-7bda-49ca-8fa5-b6a3643b56a8 admin admin] The token cache is being invalidate because project 54a56eefb1cd45a196cdf268c09a0f91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:30:35.191213 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-14851681-7bda-49ca-8fa5-b6a3643b56a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:30:35.191862 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-14851681-7bda-49ca-8fa5-b6a3643b56a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:30:35.193350 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1267/2519] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:30:34 2022] DELETE /identity/v3/projects/54a56eefb1cd45a196cdf268c09a0f91 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:30:35.198387 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def5eb92-96df-40c4-ba33-2a588dfc9551 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:35.198901 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def5eb92-96df-40c4-ba33-2a588dfc9551 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:35.199011 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def5eb92-96df-40c4-ba33-2a588dfc9551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:35.219941 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-def5eb92-96df-40c4-ba33-2a588dfc9551 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:30:35.221286 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-def5eb92-96df-40c4-ba33-2a588dfc9551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:30:35.281954 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1253/2520] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:30:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:30:35.290062 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4926ab86-d7d4-4a6b-a8bc-0f7e6f5005ad None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:35.290385 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4926ab86-d7d4-4a6b-a8bc-0f7e6f5005ad None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:35.290673 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4926ab86-d7d4-4a6b-a8bc-0f7e6f5005ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:35.311616 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-4926ab86-d7d4-4a6b-a8bc-0f7e6f5005ad None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:30:35.313455 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-4926ab86-d7d4-4a6b-a8bc-0f7e6f5005ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:30:35.321774 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1268/2521] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:30:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:30:35.326187 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def5eb92-96df-40c4-ba33-2a588dfc9551 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:30:35.336291 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def5eb92-96df-40c4-ba33-2a588dfc9551 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:30:35.341077 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94dcee8-ffd8-446b-8b58-3f5409ca657a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:30:35.341685 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94dcee8-ffd8-446b-8b58-3f5409ca657a admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:35.341969 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94dcee8-ffd8-446b-8b58-3f5409ca657a admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:35.342244 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94dcee8-ffd8-446b-8b58-3f5409ca657a admin admin] PATH_INFO: `/v3/domains` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:35.342946 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94dcee8-ffd8-446b-8b58-3f5409ca657a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:30:35.343251 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94dcee8-ffd8-446b-8b58-3f5409ca657a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:30:35.345025 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94dcee8-ffd8-446b-8b58-3f5409ca657a admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:30:35.358322 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1254/2522] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Aug 25 03:30:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:30:35.362620 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4926ab86-d7d4-4a6b-a8bc-0f7e6f5005ad None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:30:35.373003 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4926ab86-d7d4-4a6b-a8bc-0f7e6f5005ad None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:30:35.378800 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b96f5ea-0325-49d4-a642-90f32dc0d61b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:30:35.379415 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b96f5ea-0325-49d4-a642-90f32dc0d61b admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:35.379697 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b96f5ea-0325-49d4-a642-90f32dc0d61b admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:35.379973 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b96f5ea-0325-49d4-a642-90f32dc0d61b admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:35.380621 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b96f5ea-0325-49d4-a642-90f32dc0d61b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:30:35.382607 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b96f5ea-0325-49d4-a642-90f32dc0d61b admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:30:35.472201 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1269/2523] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:30:35 2022] POST /identity/v3/projects => generated 387 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:30:35.473714 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94dcee8-ffd8-446b-8b58-3f5409ca657a admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:30:35.485181 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94dcee8-ffd8-446b-8b58-3f5409ca657a admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:30:35.511876 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:30:35.542482 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2c1539-f4ed-4724-8d27-6974571303cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:30:35.543095 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2c1539-f4ed-4724-8d27-6974571303cd admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:35.543377 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2c1539-f4ed-4724-8d27-6974571303cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:35.543650 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2c1539-f4ed-4724-8d27-6974571303cd admin admin] PATH_INFO: `/v3/users` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:35.544283 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2c1539-f4ed-4724-8d27-6974571303cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:30:35.546206 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2c1539-f4ed-4724-8d27-6974571303cd admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:30:35.653654 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1255/2524] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:30:35 2022] POST /identity/v3/users => generated 294 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:30:35.654938 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b96f5ea-0325-49d4-a642-90f32dc0d61b admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:30:35.666089 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b96f5ea-0325-49d4-a642-90f32dc0d61b admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:30:35.670966 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f23b65-4cea-4974-be6f-5bd63dde886f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:30:35.671567 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f23b65-4cea-4974-be6f-5bd63dde886f admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:35.671847 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f23b65-4cea-4974-be6f-5bd63dde886f admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:35.672680 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f23b65-4cea-4974-be6f-5bd63dde886f admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:35.673327 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f23b65-4cea-4974-be6f-5bd63dde886f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:30:35.674960 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f23b65-4cea-4974-be6f-5bd63dde886f admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:30:35.685850 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1270/2525] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:30:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:30:35.690019 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2c1539-f4ed-4724-8d27-6974571303cd admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:30:35.700793 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2c1539-f4ed-4724-8d27-6974571303cd admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:30:35.706067 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9e635e-306b-4f06-877b-d581b6a6c92a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:30:35.706710 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9e635e-306b-4f06-877b-d581b6a6c92a admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:35.706988 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9e635e-306b-4f06-877b-d581b6a6c92a admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:35.707261 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9e635e-306b-4f06-877b-d581b6a6c92a admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:35.707909 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9e635e-306b-4f06-877b-d581b6a6c92a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:30:35.709559 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9e635e-306b-4f06-877b-d581b6a6c92a admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:30:35.720853 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1256/2526] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:30:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:30:35.725269 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f23b65-4cea-4974-be6f-5bd63dde886f admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:30:35.736290 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f23b65-4cea-4974-be6f-5bd63dde886f admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:30:35.741431 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cda3ff-2881-40c3-ba1d-e1492f31edb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:30:35.742102 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cda3ff-2881-40c3-ba1d-e1492f31edb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:35.742419 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cda3ff-2881-40c3-ba1d-e1492f31edb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:35.742696 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cda3ff-2881-40c3-ba1d-e1492f31edb6 admin admin] PATH_INFO: `/v3/projects/6cbeb9cd3cee4e51a211e25243485de2/users/38bff27bc9f94015a2eceb06f1318689/roles/3255f77192524a16bc405173106871fc` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:35.758553 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cda3ff-2881-40c3-ba1d-e1492f31edb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cbeb9cd3cee4e51a211e25243485de2, user_id=38bff27bc9f94015a2eceb06f1318689, role_id=3255f77192524a16bc405173106871fc)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:30:35.761268 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cda3ff-2881-40c3-ba1d-e1492f31edb6 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:30:35.869562 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1271/2527] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:30:35 2022] PUT /identity/v3/projects/6cbeb9cd3cee4e51a211e25243485de2/users/38bff27bc9f94015a2eceb06f1318689/roles/3255f77192524a16bc405173106871fc => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:30:35.881951 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662fdb44-069c-4962-a59c-0c1da4c5248e None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:35.882515 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662fdb44-069c-4962-a59c-0c1da4c5248e None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:35.882818 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662fdb44-069c-4962-a59c-0c1da4c5248e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:35.926401 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-662fdb44-069c-4962-a59c-0c1da4c5248e None None] MFA Rules not processed for user `38bff27bc9f94015a2eceb06f1318689`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:30:35.927729 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-662fdb44-069c-4962-a59c-0c1da4c5248e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:30:35.994186 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1257/2528] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:30:35 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:30:35.998489 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cda3ff-2881-40c3-ba1d-e1492f31edb6 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:30:36.063089 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cda3ff-2881-40c3-ba1d-e1492f31edb6 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:30:36.067788 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabdbed6-6dd2-4331-a4f9-c28db8d511fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:30:36.068398 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabdbed6-6dd2-4331-a4f9-c28db8d511fa admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:36.068691 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabdbed6-6dd2-4331-a4f9-c28db8d511fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:36.068948 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabdbed6-6dd2-4331-a4f9-c28db8d511fa admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:36.069571 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabdbed6-6dd2-4331-a4f9-c28db8d511fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:30:36.071517 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabdbed6-6dd2-4331-a4f9-c28db8d511fa admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:30:36.186670 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1272/2529] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:30:35 2022] POST /identity/v3/projects => generated 387 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:30:36.192599 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662fdb44-069c-4962-a59c-0c1da4c5248e None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:30:36.205531 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662fdb44-069c-4962-a59c-0c1da4c5248e None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:30:36.264996 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15d6b3d-2918-447a-8f19-8996efb6d56a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:30:36.265552 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15d6b3d-2918-447a-8f19-8996efb6d56a admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:36.266395 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15d6b3d-2918-447a-8f19-8996efb6d56a admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:36.272513 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15d6b3d-2918-447a-8f19-8996efb6d56a admin admin] PATH_INFO: `/v3/users` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:36.272513 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15d6b3d-2918-447a-8f19-8996efb6d56a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:30:36.272513 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15d6b3d-2918-447a-8f19-8996efb6d56a admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:30:37.244176 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1258/2530] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:30:36 2022] POST /identity/v3/users => generated 300 bytes in 1041 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:30:37.247262 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabdbed6-6dd2-4331-a4f9-c28db8d511fa admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:30:37.258248 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabdbed6-6dd2-4331-a4f9-c28db8d511fa admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:30:37.263226 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237e1a71-60ec-415b-bb77-2cbfb78553c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:30:37.263990 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237e1a71-60ec-415b-bb77-2cbfb78553c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:37.264270 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237e1a71-60ec-415b-bb77-2cbfb78553c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:37.264543 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237e1a71-60ec-415b-bb77-2cbfb78553c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:37.265216 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237e1a71-60ec-415b-bb77-2cbfb78553c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:30:37.267151 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237e1a71-60ec-415b-bb77-2cbfb78553c0 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:30:37.279099 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1273/2531] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:30:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:30:37.283502 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15d6b3d-2918-447a-8f19-8996efb6d56a admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:30:37.293992 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15d6b3d-2918-447a-8f19-8996efb6d56a admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:30:37.299394 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f1f49d-d761-43e5-bd58-66e489e9b4f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:30:37.300213 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f1f49d-d761-43e5-bd58-66e489e9b4f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:37.300505 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f1f49d-d761-43e5-bd58-66e489e9b4f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:37.300786 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f1f49d-d761-43e5-bd58-66e489e9b4f1 admin admin] PATH_INFO: `/v3/projects/9904edf8e37848b19f59746f8a02c9d6/users/efdc3be8ef8741b5ab0b585d050cb100/roles/62f19ea272b447ccaab1576cd9bf4fd9` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:37.316312 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f1f49d-d761-43e5-bd58-66e489e9b4f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9904edf8e37848b19f59746f8a02c9d6, user_id=efdc3be8ef8741b5ab0b585d050cb100, role_id=62f19ea272b447ccaab1576cd9bf4fd9)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:30:37.319008 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f1f49d-d761-43e5-bd58-66e489e9b4f1 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:30:37.376220 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1259/2532] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:30:37 2022] PUT /identity/v3/projects/9904edf8e37848b19f59746f8a02c9d6/users/efdc3be8ef8741b5ab0b585d050cb100/roles/62f19ea272b447ccaab1576cd9bf4fd9 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:30:37.380180 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237e1a71-60ec-415b-bb77-2cbfb78553c0 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:30:37.390731 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237e1a71-60ec-415b-bb77-2cbfb78553c0 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:30:37.413699 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:30:37.451148 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758834ae-ab6d-4b0a-9e1b-0575f31a16a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:30:37.451763 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758834ae-ab6d-4b0a-9e1b-0575f31a16a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:37.452047 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758834ae-ab6d-4b0a-9e1b-0575f31a16a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:37.452325 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758834ae-ab6d-4b0a-9e1b-0575f31a16a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:37.452970 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758834ae-ab6d-4b0a-9e1b-0575f31a16a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:30:37.454612 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758834ae-ab6d-4b0a-9e1b-0575f31a16a3 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:30:37.465148 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1274/2533] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:30:37 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:30:37.469437 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f1f49d-d761-43e5-bd58-66e489e9b4f1 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:30:37.475687 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:30:37.481342 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f1f49d-d761-43e5-bd58-66e489e9b4f1 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:30:37.486363 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c5d046-c306-4c59-8e75-0a2b6d7e95b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:30:37.487175 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c5d046-c306-4c59-8e75-0a2b6d7e95b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:37.487456 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c5d046-c306-4c59-8e75-0a2b6d7e95b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:37.487733 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c5d046-c306-4c59-8e75-0a2b6d7e95b9 admin admin] PATH_INFO: `/v3/projects/9904edf8e37848b19f59746f8a02c9d6/users/efdc3be8ef8741b5ab0b585d050cb100/roles/62f19ea272b447ccaab1576cd9bf4fd9` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:37.489742 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c5d046-c306-4c59-8e75-0a2b6d7e95b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9904edf8e37848b19f59746f8a02c9d6, user_id=efdc3be8ef8741b5ab0b585d050cb100, role_id=62f19ea272b447ccaab1576cd9bf4fd9)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:30:37.492460 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c5d046-c306-4c59-8e75-0a2b6d7e95b9 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:30:37.503961 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1260/2534] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:30:37 2022] PUT /identity/v3/projects/9904edf8e37848b19f59746f8a02c9d6/users/efdc3be8ef8741b5ab0b585d050cb100/roles/62f19ea272b447ccaab1576cd9bf4fd9 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:30:37.513986 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c44a84e-7123-4605-9ee1-30afd4adcb44 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:37.514362 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c44a84e-7123-4605-9ee1-30afd4adcb44 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:37.514646 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c44a84e-7123-4605-9ee1-30afd4adcb44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:37.545627 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-0c44a84e-7123-4605-9ee1-30afd4adcb44 None None] MFA Rules not processed for user `efdc3be8ef8741b5ab0b585d050cb100`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:30:37.546836 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-0c44a84e-7123-4605-9ee1-30afd4adcb44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:30:37.621722 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1275/2535] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:30:37 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:30:37.633824 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c5d046-c306-4c59-8e75-0a2b6d7e95b9 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:30:37.635693 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-69c5d046-c306-4c59-8e75-0a2b6d7e95b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:30:37.710699 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c5d046-c306-4c59-8e75-0a2b6d7e95b9 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:30:37.714420 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb2df55-badf-436d-92b6-13818ba2ed29 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60193ba9e1c14df3bc9eebcc6f28f2d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:30:37.715092 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb2df55-badf-436d-92b6-13818ba2ed29 service cinder] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:37.715369 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb2df55-badf-436d-92b6-13818ba2ed29 service cinder] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:37.715756 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb2df55-badf-436d-92b6-13818ba2ed29 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:37.725802 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb2df55-badf-436d-92b6-13818ba2ed29 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:30:37.727258 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb2df55-badf-436d-92b6-13818ba2ed29 service cinder] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:30:37.737290 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1261/2536] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:30:37 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:30:37.808868 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c44a84e-7123-4605-9ee1-30afd4adcb44 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:30:37.875419 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c44a84e-7123-4605-9ee1-30afd4adcb44 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:30:37.882932 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc3eaf1-24e0-4fa7-ae77-632b1c91681b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:30:37.883570 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc3eaf1-24e0-4fa7-ae77-632b1c91681b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:37.883848 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc3eaf1-24e0-4fa7-ae77-632b1c91681b admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:37.884121 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc3eaf1-24e0-4fa7-ae77-632b1c91681b admin admin] PATH_INFO: `/v3/users/38bff27bc9f94015a2eceb06f1318689` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:37.885240 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc3eaf1-24e0-4fa7-ae77-632b1c91681b admin admin] RBAC: Authorizing `identity:delete_user(user_id=38bff27bc9f94015a2eceb06f1318689)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:30:37.887265 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc3eaf1-24e0-4fa7-ae77-632b1c91681b admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:30:38.077517 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-0dc3eaf1-24e0-4fa7-ae77-632b1c91681b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38bff27bc9f94015a2eceb06f1318689'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:30:38.079307 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:30:38.136383 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-0dc3eaf1-24e0-4fa7-ae77-632b1c91681b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38bff27bc9f94015a2eceb06f1318689'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:30:38.137203 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-0dc3eaf1-24e0-4fa7-ae77-632b1c91681b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38bff27bc9f94015a2eceb06f1318689'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:30:38.148462 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-0dc3eaf1-24e0-4fa7-ae77-632b1c91681b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38bff27bc9f94015a2eceb06f1318689'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:30:38.165990 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-0dc3eaf1-24e0-4fa7-ae77-632b1c91681b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38bff27bc9f94015a2eceb06f1318689'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:30:38.172561 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1276/2537] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:30:37 2022] DELETE /identity/v3/users/38bff27bc9f94015a2eceb06f1318689 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:30:38.178040 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea010c42-80ff-4056-93e4-9e320e4108cb None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:38.178360 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea010c42-80ff-4056-93e4-9e320e4108cb None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:38.178640 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea010c42-80ff-4056-93e4-9e320e4108cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:38.203873 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-ea010c42-80ff-4056-93e4-9e320e4108cb None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:30:38.205103 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-ea010c42-80ff-4056-93e4-9e320e4108cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:30:38.277698 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1262/2538] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:30:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:30:38.289160 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc3eaf1-24e0-4fa7-ae77-632b1c91681b admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:30:38.291009 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-0dc3eaf1-24e0-4fa7-ae77-632b1c91681b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:30:38.357753 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc3eaf1-24e0-4fa7-ae77-632b1c91681b admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:30:38.362389 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e96022-2192-4a5a-9306-58a66dd1f700 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:30:38.363036 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e96022-2192-4a5a-9306-58a66dd1f700 service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:38.363328 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e96022-2192-4a5a-9306-58a66dd1f700 service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:38.363607 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e96022-2192-4a5a-9306-58a66dd1f700 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:38.372856 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e96022-2192-4a5a-9306-58a66dd1f700 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:30:38.374350 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e96022-2192-4a5a-9306-58a66dd1f700 service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:30:38.384930 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1277/2539] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:30:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 25 03:30:38.846110 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea010c42-80ff-4056-93e4-9e320e4108cb None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:30:38.847604 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-ea010c42-80ff-4056-93e4-9e320e4108cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:30:38.876485 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea010c42-80ff-4056-93e4-9e320e4108cb None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:30:38.881421 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27504509-43f0-495d-86f6-6395f60244ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:30:38.882313 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27504509-43f0-495d-86f6-6395f60244ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:38.882606 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27504509-43f0-495d-86f6-6395f60244ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:38.882881 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27504509-43f0-495d-86f6-6395f60244ff admin admin] PATH_INFO: `/v3/projects/6cbeb9cd3cee4e51a211e25243485de2` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:38.884031 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27504509-43f0-495d-86f6-6395f60244ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=6cbeb9cd3cee4e51a211e25243485de2)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:30:38.885955 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27504509-43f0-495d-86f6-6395f60244ff admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:30:38.993047 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-27504509-43f0-495d-86f6-6395f60244ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6cbeb9cd3cee4e51a211e25243485de2', 'request_id': 'req-27504509-43f0-495d-86f6-6395f60244ff'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:30:39.061117 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-27504509-43f0-495d-86f6-6395f60244ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6cbeb9cd3cee4e51a211e25243485de2', 'request_id': 'req-27504509-43f0-495d-86f6-6395f60244ff'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:30:39.067597 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-27504509-43f0-495d-86f6-6395f60244ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6cbeb9cd3cee4e51a211e25243485de2', 'request_id': 'req-27504509-43f0-495d-86f6-6395f60244ff'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:30:39.081103 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-27504509-43f0-495d-86f6-6395f60244ff admin admin] The token cache is being invalidate because project 6cbeb9cd3cee4e51a211e25243485de2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81995) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:30:39.081391 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-27504509-43f0-495d-86f6-6395f60244ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:30:39.082079 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-27504509-43f0-495d-86f6-6395f60244ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:30:39.083870 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1263/2540] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:30:38 2022] DELETE /identity/v3/projects/6cbeb9cd3cee4e51a211e25243485de2 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:30:39.087787 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e96022-2192-4a5a-9306-58a66dd1f700 service neutron] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:30:39.102667 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e96022-2192-4a5a-9306-58a66dd1f700 service neutron] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:30:39.162897 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84da1bd8-244d-4a66-8262-ce243ebdbb8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:30:39.163526 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84da1bd8-244d-4a66-8262-ce243ebdbb8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:39.163810 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84da1bd8-244d-4a66-8262-ce243ebdbb8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:39.164185 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84da1bd8-244d-4a66-8262-ce243ebdbb8b admin admin] PATH_INFO: `/v3/users/efdc3be8ef8741b5ab0b585d050cb100` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:39.165335 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84da1bd8-244d-4a66-8262-ce243ebdbb8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=efdc3be8ef8741b5ab0b585d050cb100)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:30:39.167300 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84da1bd8-244d-4a66-8262-ce243ebdbb8b admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:30:39.391388 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-84da1bd8-244d-4a66-8262-ce243ebdbb8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'efdc3be8ef8741b5ab0b585d050cb100'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:30:39.461920 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-84da1bd8-244d-4a66-8262-ce243ebdbb8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'efdc3be8ef8741b5ab0b585d050cb100'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:30:39.462910 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-84da1bd8-244d-4a66-8262-ce243ebdbb8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'efdc3be8ef8741b5ab0b585d050cb100'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:30:39.471371 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-84da1bd8-244d-4a66-8262-ce243ebdbb8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'efdc3be8ef8741b5ab0b585d050cb100'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:30:39.477585 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:INSERT=2,keystone:DELETE=12,keystone:SELECT=86 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:30:39.485906 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-84da1bd8-244d-4a66-8262-ce243ebdbb8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'efdc3be8ef8741b5ab0b585d050cb100'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:30:39.493282 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1278/2541] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:30:39 2022] DELETE /identity/v3/users/efdc3be8ef8741b5ab0b585d050cb100 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:30:39.921816 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27504509-43f0-495d-86f6-6395f60244ff admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:30:39.921816 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-27504509-43f0-495d-86f6-6395f60244ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:30:39.936258 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27504509-43f0-495d-86f6-6395f60244ff admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:30:39.958242 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:30:40.005286 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa59d56-ca42-4e56-b465-f05bb9f284a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:30:40.005902 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa59d56-ca42-4e56-b465-f05bb9f284a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:40.006182 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa59d56-ca42-4e56-b465-f05bb9f284a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:40.006487 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa59d56-ca42-4e56-b465-f05bb9f284a7 admin admin] PATH_INFO: `/v3/projects/9904edf8e37848b19f59746f8a02c9d6` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:40.007594 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa59d56-ca42-4e56-b465-f05bb9f284a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9904edf8e37848b19f59746f8a02c9d6)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:30:40.009534 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa59d56-ca42-4e56-b465-f05bb9f284a7 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:30:40.107033 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-afa59d56-ca42-4e56-b465-f05bb9f284a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9904edf8e37848b19f59746f8a02c9d6', 'request_id': 'req-afa59d56-ca42-4e56-b465-f05bb9f284a7'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:30:40.164034 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-afa59d56-ca42-4e56-b465-f05bb9f284a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9904edf8e37848b19f59746f8a02c9d6', 'request_id': 'req-afa59d56-ca42-4e56-b465-f05bb9f284a7'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:30:40.170191 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-afa59d56-ca42-4e56-b465-f05bb9f284a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9904edf8e37848b19f59746f8a02c9d6', 'request_id': 'req-afa59d56-ca42-4e56-b465-f05bb9f284a7'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:30:40.181374 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-afa59d56-ca42-4e56-b465-f05bb9f284a7 admin admin] The token cache is being invalidate because project 9904edf8e37848b19f59746f8a02c9d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81995) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:30:40.181630 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-afa59d56-ca42-4e56-b465-f05bb9f284a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:30:40.182254 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-afa59d56-ca42-4e56-b465-f05bb9f284a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:30:40.183934 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1264/2542] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:30:39 2022] DELETE /identity/v3/projects/9904edf8e37848b19f59746f8a02c9d6 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:30:40.190475 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8f12c0-2360-4673-9a09-c24e3e2e48ad None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:40.190767 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8f12c0-2360-4673-9a09-c24e3e2e48ad None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:40.191044 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8f12c0-2360-4673-9a09-c24e3e2e48ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:40.211058 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-5e8f12c0-2360-4673-9a09-c24e3e2e48ad None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:30:40.212251 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-5e8f12c0-2360-4673-9a09-c24e3e2e48ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:30:40.272302 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1279/2543] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:30:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:30:40.280901 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704a1bca-d171-422b-972d-b0f6a1bdd274 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:40.281187 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704a1bca-d171-422b-972d-b0f6a1bdd274 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:40.281463 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704a1bca-d171-422b-972d-b0f6a1bdd274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:40.300887 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-704a1bca-d171-422b-972d-b0f6a1bdd274 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:30:40.302109 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-704a1bca-d171-422b-972d-b0f6a1bdd274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:30:40.310185 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1265/2544] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:30:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:30:40.314790 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8f12c0-2360-4673-9a09-c24e3e2e48ad None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:30:40.324457 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8f12c0-2360-4673-9a09-c24e3e2e48ad None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:30:40.329082 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fa6473-802c-4db3-b745-dab7883c0c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:30:40.329695 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fa6473-802c-4db3-b745-dab7883c0c71 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:40.330582 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fa6473-802c-4db3-b745-dab7883c0c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:40.330860 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fa6473-802c-4db3-b745-dab7883c0c71 admin admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:40.331508 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fa6473-802c-4db3-b745-dab7883c0c71 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:30:40.331813 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fa6473-802c-4db3-b745-dab7883c0c71 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:30:40.333406 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fa6473-802c-4db3-b745-dab7883c0c71 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:30:40.344656 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1280/2545] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Aug 25 03:30:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:30:40.348623 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704a1bca-d171-422b-972d-b0f6a1bdd274 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:30:40.358520 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704a1bca-d171-422b-972d-b0f6a1bdd274 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:30:40.363376 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc92fa57-e6df-4b60-8229-cb82ec51dea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:30:40.364006 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc92fa57-e6df-4b60-8229-cb82ec51dea6 admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:40.364290 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc92fa57-e6df-4b60-8229-cb82ec51dea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:40.364565 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc92fa57-e6df-4b60-8229-cb82ec51dea6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:40.365188 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc92fa57-e6df-4b60-8229-cb82ec51dea6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:30:40.367165 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc92fa57-e6df-4b60-8229-cb82ec51dea6 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:30:40.491638 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1266/2546] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:30:40 2022] POST /identity/v3/projects => generated 361 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:30:40.495725 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fa6473-802c-4db3-b745-dab7883c0c71 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:30:40.506354 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fa6473-802c-4db3-b745-dab7883c0c71 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:30:40.565257 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96cdc47-2dc9-4bc8-b882-00e76223be4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:30:40.565897 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96cdc47-2dc9-4bc8-b882-00e76223be4c admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:40.566176 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96cdc47-2dc9-4bc8-b882-00e76223be4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:40.566480 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96cdc47-2dc9-4bc8-b882-00e76223be4c admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:40.567113 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96cdc47-2dc9-4bc8-b882-00e76223be4c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:30:40.569053 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96cdc47-2dc9-4bc8-b882-00e76223be4c admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:30:40.647543 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1281/2547] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:30:40 2022] POST /identity/v3/users => generated 281 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:30:40.648985 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc92fa57-e6df-4b60-8229-cb82ec51dea6 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:30:40.659448 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc92fa57-e6df-4b60-8229-cb82ec51dea6 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:30:40.664415 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca850535-09dc-4cd2-8a6c-75ec3aa3282b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:30:40.664993 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca850535-09dc-4cd2-8a6c-75ec3aa3282b admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:40.665336 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca850535-09dc-4cd2-8a6c-75ec3aa3282b admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:40.665691 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca850535-09dc-4cd2-8a6c-75ec3aa3282b admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:40.666536 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca850535-09dc-4cd2-8a6c-75ec3aa3282b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:30:40.668281 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca850535-09dc-4cd2-8a6c-75ec3aa3282b admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:30:40.678961 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1267/2548] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:30:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 25 03:30:40.682759 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96cdc47-2dc9-4bc8-b882-00e76223be4c admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:30:40.693420 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96cdc47-2dc9-4bc8-b882-00e76223be4c admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:30:40.698012 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7517de31-6dee-4309-8cf4-5c0ed025205c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:30:40.698634 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7517de31-6dee-4309-8cf4-5c0ed025205c admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:40.698911 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7517de31-6dee-4309-8cf4-5c0ed025205c admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:40.699181 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7517de31-6dee-4309-8cf4-5c0ed025205c admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:40.699811 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7517de31-6dee-4309-8cf4-5c0ed025205c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:30:40.701420 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7517de31-6dee-4309-8cf4-5c0ed025205c admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:30:40.711121 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1282/2549] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:30:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 25 03:30:40.715198 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca850535-09dc-4cd2-8a6c-75ec3aa3282b admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:30:40.724900 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca850535-09dc-4cd2-8a6c-75ec3aa3282b admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:30:40.729389 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073a9f29-4c96-41e2-9685-7138a13d565e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:30:40.730019 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073a9f29-4c96-41e2-9685-7138a13d565e admin admin] REQUEST_METHOD: `PUT` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:40.730381 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073a9f29-4c96-41e2-9685-7138a13d565e admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:40.730660 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073a9f29-4c96-41e2-9685-7138a13d565e admin admin] PATH_INFO: `/v3/projects/67da4ec0f35147e89d43e00eb9a3da6c/users/27891cc85d0b4e05857c263c5d65519b/roles/3255f77192524a16bc405173106871fc` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:40.745215 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073a9f29-4c96-41e2-9685-7138a13d565e admin admin] RBAC: Authorizing `identity:create_grant(project_id=67da4ec0f35147e89d43e00eb9a3da6c, user_id=27891cc85d0b4e05857c263c5d65519b, role_id=3255f77192524a16bc405173106871fc)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:30:40.747887 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073a9f29-4c96-41e2-9685-7138a13d565e admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:30:40.815544 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1268/2550] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:30:40 2022] PUT /identity/v3/projects/67da4ec0f35147e89d43e00eb9a3da6c/users/27891cc85d0b4e05857c263c5d65519b/roles/3255f77192524a16bc405173106871fc => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:30:40.825959 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cf17ee-f21d-46e6-848a-97aa06586dff None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:40.826250 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cf17ee-f21d-46e6-848a-97aa06586dff None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:40.826560 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cf17ee-f21d-46e6-848a-97aa06586dff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:40.854337 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-18cf17ee-f21d-46e6-848a-97aa06586dff None None] MFA Rules not processed for user `27891cc85d0b4e05857c263c5d65519b`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:30:40.855553 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-18cf17ee-f21d-46e6-848a-97aa06586dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:30:40.899330 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:30:40.921531 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1283/2551] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:30:40 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:30:40.957706 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073a9f29-4c96-41e2-9685-7138a13d565e admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:30:40.959425 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-073a9f29-4c96-41e2-9685-7138a13d565e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:30:41.004913 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:30:41.016056 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073a9f29-4c96-41e2-9685-7138a13d565e admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:30:41.019747 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21563c2e-880d-4830-af3a-978e66ca0f9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60193ba9e1c14df3bc9eebcc6f28f2d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:30:41.020349 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21563c2e-880d-4830-af3a-978e66ca0f9e service cinder] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:41.020636 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21563c2e-880d-4830-af3a-978e66ca0f9e service cinder] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:41.020906 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21563c2e-880d-4830-af3a-978e66ca0f9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:41.028974 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21563c2e-880d-4830-af3a-978e66ca0f9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:30:41.030445 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21563c2e-880d-4830-af3a-978e66ca0f9e service cinder] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:30:41.039855 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1269/2552] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:30:40 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:30:41.056334 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cf17ee-f21d-46e6-848a-97aa06586dff None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:30:41.117644 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cf17ee-f21d-46e6-848a-97aa06586dff None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:30:41.122155 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4df01f-8d60-48ac-98fb-6c4816086e5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:30:41.122808 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4df01f-8d60-48ac-98fb-6c4816086e5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:41.123091 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4df01f-8d60-48ac-98fb-6c4816086e5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:41.123365 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4df01f-8d60-48ac-98fb-6c4816086e5f admin admin] PATH_INFO: `/v3/users/27891cc85d0b4e05857c263c5d65519b` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:41.124479 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4df01f-8d60-48ac-98fb-6c4816086e5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=27891cc85d0b4e05857c263c5d65519b)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:30:41.126431 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4df01f-8d60-48ac-98fb-6c4816086e5f admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:30:41.317624 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-0c4df01f-8d60-48ac-98fb-6c4816086e5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27891cc85d0b4e05857c263c5d65519b'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:30:41.393364 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-0c4df01f-8d60-48ac-98fb-6c4816086e5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27891cc85d0b4e05857c263c5d65519b'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:30:41.394013 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-0c4df01f-8d60-48ac-98fb-6c4816086e5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27891cc85d0b4e05857c263c5d65519b'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:30:41.402334 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-0c4df01f-8d60-48ac-98fb-6c4816086e5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27891cc85d0b4e05857c263c5d65519b'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:30:41.415655 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-0c4df01f-8d60-48ac-98fb-6c4816086e5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27891cc85d0b4e05857c263c5d65519b'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:30:41.421788 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1284/2553] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:30:41 2022] DELETE /identity/v3/users/27891cc85d0b4e05857c263c5d65519b => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:30:41.426397 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc72d2b7-2e95-4a43-a35b-2470153a143f None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:41.426791 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc72d2b7-2e95-4a43-a35b-2470153a143f None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:41.427075 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc72d2b7-2e95-4a43-a35b-2470153a143f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:41.446231 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-cc72d2b7-2e95-4a43-a35b-2470153a143f None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:30:41.447462 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-cc72d2b7-2e95-4a43-a35b-2470153a143f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:30:41.508802 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1270/2554] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:30:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:30:41.519403 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4df01f-8d60-48ac-98fb-6c4816086e5f admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:30:41.520962 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-0c4df01f-8d60-48ac-98fb-6c4816086e5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:30:41.580726 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4df01f-8d60-48ac-98fb-6c4816086e5f admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:30:41.584519 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72396dc9-e1a7-4c35-82cc-88a0ba550f73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:30:41.585137 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72396dc9-e1a7-4c35-82cc-88a0ba550f73 service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:41.585420 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72396dc9-e1a7-4c35-82cc-88a0ba550f73 service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:41.585698 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72396dc9-e1a7-4c35-82cc-88a0ba550f73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:41.595271 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72396dc9-e1a7-4c35-82cc-88a0ba550f73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:30:41.596700 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72396dc9-e1a7-4c35-82cc-88a0ba550f73 service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:30:41.606149 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1285/2555] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:30:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:30:42.069122 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc72d2b7-2e95-4a43-a35b-2470153a143f None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:30:42.070685 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-cc72d2b7-2e95-4a43-a35b-2470153a143f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:30:42.082877 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc72d2b7-2e95-4a43-a35b-2470153a143f None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:30:42.087462 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4672132-6b93-446a-9a0e-8af6caa34367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:30:42.088212 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4672132-6b93-446a-9a0e-8af6caa34367 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:42.088495 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4672132-6b93-446a-9a0e-8af6caa34367 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:42.088771 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4672132-6b93-446a-9a0e-8af6caa34367 admin admin] PATH_INFO: `/v3/projects/67da4ec0f35147e89d43e00eb9a3da6c` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:42.089904 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4672132-6b93-446a-9a0e-8af6caa34367 admin admin] RBAC: Authorizing `identity:delete_project(project_id=67da4ec0f35147e89d43e00eb9a3da6c)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:30:42.091896 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4672132-6b93-446a-9a0e-8af6caa34367 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:30:42.205073 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-f4672132-6b93-446a-9a0e-8af6caa34367 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67da4ec0f35147e89d43e00eb9a3da6c', 'request_id': 'req-f4672132-6b93-446a-9a0e-8af6caa34367'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:30:42.295727 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-f4672132-6b93-446a-9a0e-8af6caa34367 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67da4ec0f35147e89d43e00eb9a3da6c', 'request_id': 'req-f4672132-6b93-446a-9a0e-8af6caa34367'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:30:42.302630 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-f4672132-6b93-446a-9a0e-8af6caa34367 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67da4ec0f35147e89d43e00eb9a3da6c', 'request_id': 'req-f4672132-6b93-446a-9a0e-8af6caa34367'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:30:42.313968 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-f4672132-6b93-446a-9a0e-8af6caa34367 admin admin] The token cache is being invalidate because project 67da4ec0f35147e89d43e00eb9a3da6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81995) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:30:42.314294 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-f4672132-6b93-446a-9a0e-8af6caa34367 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:30:42.315005 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-f4672132-6b93-446a-9a0e-8af6caa34367 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:30:42.316495 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1271/2556] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:30:42 2022] DELETE /identity/v3/projects/67da4ec0f35147e89d43e00eb9a3da6c => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:30:42.321578 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5f5763-e8c1-4667-ac3e-3731bc339aa7 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:42.321867 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5f5763-e8c1-4667-ac3e-3731bc339aa7 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:42.322141 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5f5763-e8c1-4667-ac3e-3731bc339aa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:42.330861 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:30:42.341879 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-8c5f5763-e8c1-4667-ac3e-3731bc339aa7 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:30:42.343174 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-8c5f5763-e8c1-4667-ac3e-3731bc339aa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:30:42.403368 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1286/2557] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:30:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:30:42.411754 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209ec473-ddc3-43d4-9ac4-1f96eb3e0637 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:42.412076 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209ec473-ddc3-43d4-9ac4-1f96eb3e0637 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:42.412350 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209ec473-ddc3-43d4-9ac4-1f96eb3e0637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:42.432133 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-209ec473-ddc3-43d4-9ac4-1f96eb3e0637 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:30:42.433322 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-209ec473-ddc3-43d4-9ac4-1f96eb3e0637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:30:42.441494 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1272/2558] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:30:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:30:42.445470 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5f5763-e8c1-4667-ac3e-3731bc339aa7 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:30:42.455021 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5f5763-e8c1-4667-ac3e-3731bc339aa7 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:30:42.459615 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4211df-28ac-4dbe-ab4e-99980b4ddea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:30:42.460148 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4211df-28ac-4dbe-ab4e-99980b4ddea3 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:42.460427 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4211df-28ac-4dbe-ab4e-99980b4ddea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:42.460702 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4211df-28ac-4dbe-ab4e-99980b4ddea3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:42.461341 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4211df-28ac-4dbe-ab4e-99980b4ddea3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:30:42.461642 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4211df-28ac-4dbe-ab4e-99980b4ddea3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:30:42.463268 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4211df-28ac-4dbe-ab4e-99980b4ddea3 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:30:42.475124 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1287/2559] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Aug 25 03:30:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:30:42.478973 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209ec473-ddc3-43d4-9ac4-1f96eb3e0637 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:30:42.488763 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209ec473-ddc3-43d4-9ac4-1f96eb3e0637 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:30:42.493337 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe7f7f6-3fa1-47f3-84c3-207553385740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:30:42.493928 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe7f7f6-3fa1-47f3-84c3-207553385740 admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:42.494205 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe7f7f6-3fa1-47f3-84c3-207553385740 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:42.494512 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe7f7f6-3fa1-47f3-84c3-207553385740 admin admin] PATH_INFO: `/v3/projects` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:42.495132 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe7f7f6-3fa1-47f3-84c3-207553385740 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:30:42.497102 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe7f7f6-3fa1-47f3-84c3-207553385740 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:30:42.588095 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1273/2560] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:30:42 2022] POST /identity/v3/projects => generated 373 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:30:42.592276 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4211df-28ac-4dbe-ab4e-99980b4ddea3 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:30:42.602685 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4211df-28ac-4dbe-ab4e-99980b4ddea3 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:30:42.658268 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b011cb2c-9937-4a86-b062-0e599e0e6ae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:30:42.659634 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b011cb2c-9937-4a86-b062-0e599e0e6ae8 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:42.659952 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b011cb2c-9937-4a86-b062-0e599e0e6ae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:42.660227 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b011cb2c-9937-4a86-b062-0e599e0e6ae8 admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:42.660856 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b011cb2c-9937-4a86-b062-0e599e0e6ae8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:30:42.663114 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b011cb2c-9937-4a86-b062-0e599e0e6ae8 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:30:42.745062 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1288/2561] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:30:42 2022] POST /identity/v3/users => generated 287 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:30:42.749561 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe7f7f6-3fa1-47f3-84c3-207553385740 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:30:42.757524 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe7f7f6-3fa1-47f3-84c3-207553385740 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:30:42.762165 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0888f9-3d60-4e88-bca1-17412c6db3c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:30:42.762937 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0888f9-3d60-4e88-bca1-17412c6db3c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:42.763231 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0888f9-3d60-4e88-bca1-17412c6db3c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:42.763507 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0888f9-3d60-4e88-bca1-17412c6db3c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:42.764264 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0888f9-3d60-4e88-bca1-17412c6db3c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:30:42.765874 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0888f9-3d60-4e88-bca1-17412c6db3c4 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:30:42.779421 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1274/2562] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:30:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:30:42.780684 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b011cb2c-9937-4a86-b062-0e599e0e6ae8 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:30:42.791426 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b011cb2c-9937-4a86-b062-0e599e0e6ae8 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:30:42.796085 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2e1f8d-ac37-4a64-8975-41ae2bc69f6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:30:42.796799 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2e1f8d-ac37-4a64-8975-41ae2bc69f6f admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:42.796960 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2e1f8d-ac37-4a64-8975-41ae2bc69f6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:42.797235 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2e1f8d-ac37-4a64-8975-41ae2bc69f6f admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:42.797909 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2e1f8d-ac37-4a64-8975-41ae2bc69f6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:30:42.799539 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2e1f8d-ac37-4a64-8975-41ae2bc69f6f admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:30:42.810462 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1289/2563] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:30:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:30:42.814579 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0888f9-3d60-4e88-bca1-17412c6db3c4 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:30:42.824340 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0888f9-3d60-4e88-bca1-17412c6db3c4 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:30:42.828959 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87206345-589e-44fa-aa84-e9844cfb05ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:30:42.829612 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87206345-589e-44fa-aa84-e9844cfb05ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:42.829892 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87206345-589e-44fa-aa84-e9844cfb05ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:42.830170 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87206345-589e-44fa-aa84-e9844cfb05ab admin admin] PATH_INFO: `/v3/projects/a52f652139d7412b8ca6cce9d110fad6/users/5f5ca6b2b6434daeaf6daf9a201c240c/roles/3255f77192524a16bc405173106871fc` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:42.848178 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87206345-589e-44fa-aa84-e9844cfb05ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=a52f652139d7412b8ca6cce9d110fad6, user_id=5f5ca6b2b6434daeaf6daf9a201c240c, role_id=3255f77192524a16bc405173106871fc)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:30:42.850986 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87206345-589e-44fa-aa84-e9844cfb05ab admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:30:42.953569 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1275/2564] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:30:42 2022] PUT /identity/v3/projects/a52f652139d7412b8ca6cce9d110fad6/users/5f5ca6b2b6434daeaf6daf9a201c240c/roles/3255f77192524a16bc405173106871fc => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:30:42.965245 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fa8ebf-bf93-4795-88f7-acbc9074db15 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:42.965540 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fa8ebf-bf93-4795-88f7-acbc9074db15 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:42.965818 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fa8ebf-bf93-4795-88f7-acbc9074db15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:42.995653 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-e2fa8ebf-bf93-4795-88f7-acbc9074db15 None None] MFA Rules not processed for user `5f5ca6b2b6434daeaf6daf9a201c240c`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:30:42.996349 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-e2fa8ebf-bf93-4795-88f7-acbc9074db15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:30:43.071745 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:30:43.075533 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1290/2565] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:30:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:30:43.086997 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87206345-589e-44fa-aa84-e9844cfb05ab admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:30:43.088706 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-87206345-589e-44fa-aa84-e9844cfb05ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:30:43.095482 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:30:43.146770 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87206345-589e-44fa-aa84-e9844cfb05ab admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:30:43.150470 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1812be14-2187-4286-a217-964bca18d5f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60193ba9e1c14df3bc9eebcc6f28f2d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:30:43.151077 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1812be14-2187-4286-a217-964bca18d5f1 service cinder] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:43.151352 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1812be14-2187-4286-a217-964bca18d5f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:43.151623 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1812be14-2187-4286-a217-964bca18d5f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:43.160236 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1812be14-2187-4286-a217-964bca18d5f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:30:43.161671 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1812be14-2187-4286-a217-964bca18d5f1 service cinder] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:30:43.170793 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1276/2566] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:30:43 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:30:45.867127 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2731da85-0256-4157-925b-d0f88d611860 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:45.867719 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2731da85-0256-4157-925b-d0f88d611860 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:45.867719 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2731da85-0256-4157-925b-d0f88d611860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:45.887931 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-2731da85-0256-4157-925b-d0f88d611860 None None] MFA Rules not processed for user `5f5ca6b2b6434daeaf6daf9a201c240c`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:30:45.889072 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-2731da85-0256-4157-925b-d0f88d611860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:30:45.897285 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1291/2567] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:30:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:30:47.323874 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e819272c-46ce-420d-bd52-b69a70b162cf None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:47.324431 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e819272c-46ce-420d-bd52-b69a70b162cf None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:47.324431 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e819272c-46ce-420d-bd52-b69a70b162cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:47.343725 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-e819272c-46ce-420d-bd52-b69a70b162cf None None] MFA Rules not processed for user `712c35b765b64919aa3eaaf02892590d`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:30:47.344912 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-e819272c-46ce-420d-bd52-b69a70b162cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:30:47.400027 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1277/2568] 10.222.0.58 () {58 vars in 967 bytes} [Thu Aug 25 03:30:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:30:47.413403 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2731da85-0256-4157-925b-d0f88d611860 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:30:47.415031 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-2731da85-0256-4157-925b-d0f88d611860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:30:47.482929 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2731da85-0256-4157-925b-d0f88d611860 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:30:47.488114 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b03aa0-1eed-4b25-8adb-450a54a58b09 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5047e1a7cd2d416d90b2d8c5a6596522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:30:47.488726 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b03aa0-1eed-4b25-8adb-450a54a58b09 service swift] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:47.489003 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b03aa0-1eed-4b25-8adb-450a54a58b09 service swift] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:47.489275 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b03aa0-1eed-4b25-8adb-450a54a58b09 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:47.497467 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b03aa0-1eed-4b25-8adb-450a54a58b09 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:30:47.499132 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b03aa0-1eed-4b25-8adb-450a54a58b09 service swift] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:30:47.507612 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1292/2569] 10.222.0.58 () {60 vars in 1384 bytes} [Thu Aug 25 03:30:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 25 03:30:50.808149 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e819272c-46ce-420d-bd52-b69a70b162cf None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:30:50.810021 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-e819272c-46ce-420d-bd52-b69a70b162cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:30:50.887756 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e819272c-46ce-420d-bd52-b69a70b162cf None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:30:50.893934 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8fc526-8c9c-4d67-bde5-8d03bf9f626f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cd171bf4df94f8fab397e42835feeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:30:50.894594 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8fc526-8c9c-4d67-bde5-8d03bf9f626f service nova] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:30:50.894877 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8fc526-8c9c-4d67-bde5-8d03bf9f626f service nova] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:30:50.895154 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8fc526-8c9c-4d67-bde5-8d03bf9f626f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:30:50.896911 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-8d8fc526-8c9c-4d67-bde5-8d03bf9f626f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:30:50.909491 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8fc526-8c9c-4d67-bde5-8d03bf9f626f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:30:50.911215 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8fc526-8c9c-4d67-bde5-8d03bf9f626f service nova] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:30:50.922785 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1278/2570] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:30:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:30:57.504382 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=35 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:31:00.918634 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=83 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:31:04.854369 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab7063c-1752-4077-86d0-afd4e5bbd4fb None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:31:04.854841 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab7063c-1752-4077-86d0-afd4e5bbd4fb None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:31:04.854960 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab7063c-1752-4077-86d0-afd4e5bbd4fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:31:04.875502 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-2ab7063c-1752-4077-86d0-afd4e5bbd4fb None None] MFA Rules not processed for user `712c35b765b64919aa3eaaf02892590d`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:31:04.876756 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-2ab7063c-1752-4077-86d0-afd4e5bbd4fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:31:04.884145 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1293/2571] 10.222.0.58 () {58 vars in 1009 bytes} [Thu Aug 25 03:31:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:31:04.905228 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8fc526-8c9c-4d67-bde5-8d03bf9f626f service nova] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:31:04.916891 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8fc526-8c9c-4d67-bde5-8d03bf9f626f service nova] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:31:04.922042 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae6e028-b781-4d9a-9599-44c92cb0f0b2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5047e1a7cd2d416d90b2d8c5a6596522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:31:04.922776 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae6e028-b781-4d9a-9599-44c92cb0f0b2 service swift] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:31:04.923058 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae6e028-b781-4d9a-9599-44c92cb0f0b2 service swift] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:31:04.923333 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae6e028-b781-4d9a-9599-44c92cb0f0b2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:31:04.932532 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae6e028-b781-4d9a-9599-44c92cb0f0b2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:31:04.934461 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae6e028-b781-4d9a-9599-44c92cb0f0b2 service swift] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:31:04.942944 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1279/2572] 10.222.0.58 () {60 vars in 1384 bytes} [Thu Aug 25 03:31:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 25 03:31:05.756095 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5576f5c-8148-4d36-b2cf-74ecdd4521ae None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:31:05.756409 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5576f5c-8148-4d36-b2cf-74ecdd4521ae None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:31:05.756692 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5576f5c-8148-4d36-b2cf-74ecdd4521ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:31:05.776447 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-e5576f5c-8148-4d36-b2cf-74ecdd4521ae None None] MFA Rules not processed for user `712c35b765b64919aa3eaaf02892590d`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:31:05.777650 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-e5576f5c-8148-4d36-b2cf-74ecdd4521ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:31:05.784773 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1294/2573] 10.222.0.58 () {58 vars in 967 bytes} [Thu Aug 25 03:31:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:31:05.796956 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae6e028-b781-4d9a-9599-44c92cb0f0b2 service swift] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:31:05.807673 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae6e028-b781-4d9a-9599-44c92cb0f0b2 service swift] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:31:05.812960 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7555acb1-c5d6-420b-a976-1efca04693ea service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5047e1a7cd2d416d90b2d8c5a6596522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:31:05.813554 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7555acb1-c5d6-420b-a976-1efca04693ea service swift] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:31:05.813835 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7555acb1-c5d6-420b-a976-1efca04693ea service swift] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:31:05.814111 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7555acb1-c5d6-420b-a976-1efca04693ea service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:31:05.822586 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7555acb1-c5d6-420b-a976-1efca04693ea service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:31:05.824157 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7555acb1-c5d6-420b-a976-1efca04693ea service swift] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:31:05.832725 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1280/2574] 10.222.0.58 () {60 vars in 1384 bytes} [Thu Aug 25 03:31:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 25 03:31:06.453798 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7af5215-910d-404e-9317-8e71473f6605 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:31:06.454319 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7af5215-910d-404e-9317-8e71473f6605 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:31:06.454438 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7af5215-910d-404e-9317-8e71473f6605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:31:06.472900 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-a7af5215-910d-404e-9317-8e71473f6605 None None] MFA Rules not processed for user `712c35b765b64919aa3eaaf02892590d`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:31:06.474630 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-a7af5215-910d-404e-9317-8e71473f6605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:31:06.482035 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1295/2575] 10.222.0.58 () {58 vars in 1009 bytes} [Thu Aug 25 03:31:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:31:06.495017 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7555acb1-c5d6-420b-a976-1efca04693ea service swift] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:31:06.505528 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7555acb1-c5d6-420b-a976-1efca04693ea service swift] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:31:06.510530 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265fe9ea-8f9a-44a5-8051-30f0de111587 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5047e1a7cd2d416d90b2d8c5a6596522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:31:06.511123 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265fe9ea-8f9a-44a5-8051-30f0de111587 service swift] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:31:06.511406 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265fe9ea-8f9a-44a5-8051-30f0de111587 service swift] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:31:06.511682 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265fe9ea-8f9a-44a5-8051-30f0de111587 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:31:06.519940 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265fe9ea-8f9a-44a5-8051-30f0de111587 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:31:06.521541 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265fe9ea-8f9a-44a5-8051-30f0de111587 service swift] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:31:06.529954 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1281/2576] 10.222.0.58 () {60 vars in 1384 bytes} [Thu Aug 25 03:31:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 25 03:31:07.142818 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1402ffa-6b89-40c5-b7f6-0cf27388e488 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:31:07.142818 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1402ffa-6b89-40c5-b7f6-0cf27388e488 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:31:07.142818 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1402ffa-6b89-40c5-b7f6-0cf27388e488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:31:07.161929 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-b1402ffa-6b89-40c5-b7f6-0cf27388e488 None None] MFA Rules not processed for user `712c35b765b64919aa3eaaf02892590d`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:31:07.165615 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-b1402ffa-6b89-40c5-b7f6-0cf27388e488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:31:07.170437 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1296/2577] 10.222.0.58 () {58 vars in 967 bytes} [Thu Aug 25 03:31:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:31:07.183133 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265fe9ea-8f9a-44a5-8051-30f0de111587 service swift] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:31:07.194754 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265fe9ea-8f9a-44a5-8051-30f0de111587 service swift] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:31:07.200228 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38272eed-00fb-408d-b4b8-8aec95632afe service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5047e1a7cd2d416d90b2d8c5a6596522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:31:07.200825 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38272eed-00fb-408d-b4b8-8aec95632afe service swift] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:31:07.201161 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38272eed-00fb-408d-b4b8-8aec95632afe service swift] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:31:07.201482 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38272eed-00fb-408d-b4b8-8aec95632afe service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:31:07.211159 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38272eed-00fb-408d-b4b8-8aec95632afe service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:31:07.212774 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38272eed-00fb-408d-b4b8-8aec95632afe service swift] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:31:07.221144 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1282/2578] 10.222.0.58 () {60 vars in 1384 bytes} [Thu Aug 25 03:31:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 25 03:31:09.774886 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1402ffa-6b89-40c5-b7f6-0cf27388e488 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:31:09.836542 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1402ffa-6b89-40c5-b7f6-0cf27388e488 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:31:09.841072 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cedf645-fa53-4f31-9b44-e838ad1e8c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:31:09.841695 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cedf645-fa53-4f31-9b44-e838ad1e8c85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:31:09.841979 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cedf645-fa53-4f31-9b44-e838ad1e8c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:31:09.842256 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cedf645-fa53-4f31-9b44-e838ad1e8c85 admin admin] PATH_INFO: `/v3/users/5f5ca6b2b6434daeaf6daf9a201c240c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:31:09.843415 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cedf645-fa53-4f31-9b44-e838ad1e8c85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f5ca6b2b6434daeaf6daf9a201c240c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:31:09.845380 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cedf645-fa53-4f31-9b44-e838ad1e8c85 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:31:10.005588 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-9cedf645-fa53-4f31-9b44-e838ad1e8c85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f5ca6b2b6434daeaf6daf9a201c240c'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:31:10.066192 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-9cedf645-fa53-4f31-9b44-e838ad1e8c85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f5ca6b2b6434daeaf6daf9a201c240c'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:31:10.067370 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-9cedf645-fa53-4f31-9b44-e838ad1e8c85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f5ca6b2b6434daeaf6daf9a201c240c'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:31:10.074577 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-9cedf645-fa53-4f31-9b44-e838ad1e8c85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f5ca6b2b6434daeaf6daf9a201c240c'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:31:10.086946 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-9cedf645-fa53-4f31-9b44-e838ad1e8c85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f5ca6b2b6434daeaf6daf9a201c240c'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:31:10.092768 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1297/2579] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:31:09 2022] DELETE /identity/v3/users/5f5ca6b2b6434daeaf6daf9a201c240c => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:31:10.097329 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5566fe3-1a07-4729-9a81-b047d9805180 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:31:10.097785 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5566fe3-1a07-4729-9a81-b047d9805180 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:31:10.097900 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5566fe3-1a07-4729-9a81-b047d9805180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:31:10.116473 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-d5566fe3-1a07-4729-9a81-b047d9805180 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:31:10.117682 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-d5566fe3-1a07-4729-9a81-b047d9805180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:31:10.174924 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1283/2580] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:31:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:31:10.185091 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cedf645-fa53-4f31-9b44-e838ad1e8c85 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:31:10.186535 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-9cedf645-fa53-4f31-9b44-e838ad1e8c85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:31:10.243553 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cedf645-fa53-4f31-9b44-e838ad1e8c85 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:31:10.247183 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46df4100-5778-4a83-993c-069603153c53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:31:10.247784 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46df4100-5778-4a83-993c-069603153c53 service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:31:10.248067 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46df4100-5778-4a83-993c-069603153c53 service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:31:10.248340 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46df4100-5778-4a83-993c-069603153c53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:31:10.255576 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:31:10.257944 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46df4100-5778-4a83-993c-069603153c53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:31:10.259352 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46df4100-5778-4a83-993c-069603153c53 service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:31:10.268851 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1298/2581] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:31:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:31:10.484470 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5566fe3-1a07-4729-9a81-b047d9805180 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:31:10.485889 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-d5566fe3-1a07-4729-9a81-b047d9805180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:31:10.500253 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5566fe3-1a07-4729-9a81-b047d9805180 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:31:10.505525 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ad7670-0222-4a1d-b520-a6e0abb2a3c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:31:10.506175 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ad7670-0222-4a1d-b520-a6e0abb2a3c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:31:10.506496 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ad7670-0222-4a1d-b520-a6e0abb2a3c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:31:10.506768 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ad7670-0222-4a1d-b520-a6e0abb2a3c0 admin admin] PATH_INFO: `/v3/projects/a52f652139d7412b8ca6cce9d110fad6` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:31:10.507902 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ad7670-0222-4a1d-b520-a6e0abb2a3c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a52f652139d7412b8ca6cce9d110fad6)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:31:10.509798 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ad7670-0222-4a1d-b520-a6e0abb2a3c0 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:31:10.614512 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-49ad7670-0222-4a1d-b520-a6e0abb2a3c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a52f652139d7412b8ca6cce9d110fad6', 'request_id': 'req-49ad7670-0222-4a1d-b520-a6e0abb2a3c0'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:31:10.689151 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-49ad7670-0222-4a1d-b520-a6e0abb2a3c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a52f652139d7412b8ca6cce9d110fad6', 'request_id': 'req-49ad7670-0222-4a1d-b520-a6e0abb2a3c0'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:31:10.695174 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-49ad7670-0222-4a1d-b520-a6e0abb2a3c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a52f652139d7412b8ca6cce9d110fad6', 'request_id': 'req-49ad7670-0222-4a1d-b520-a6e0abb2a3c0'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:31:10.706492 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-49ad7670-0222-4a1d-b520-a6e0abb2a3c0 admin admin] The token cache is being invalidate because project a52f652139d7412b8ca6cce9d110fad6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81995) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:31:10.706787 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-49ad7670-0222-4a1d-b520-a6e0abb2a3c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:31:10.707422 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-49ad7670-0222-4a1d-b520-a6e0abb2a3c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:31:10.709008 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1284/2582] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:31:10 2022] DELETE /identity/v3/projects/a52f652139d7412b8ca6cce9d110fad6 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:31:10.713899 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c222e9e-88d9-43d0-ab62-c1f58adcb3b0 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:31:10.714187 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c222e9e-88d9-43d0-ab62-c1f58adcb3b0 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:31:10.714515 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c222e9e-88d9-43d0-ab62-c1f58adcb3b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:31:10.734262 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-2c222e9e-88d9-43d0-ab62-c1f58adcb3b0 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:31:10.735476 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-2c222e9e-88d9-43d0-ab62-c1f58adcb3b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:31:10.793298 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1299/2583] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:31:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:31:10.801029 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9a28fe-5d69-43bd-afad-26658905dba8 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:31:10.801344 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9a28fe-5d69-43bd-afad-26658905dba8 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:31:10.801621 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9a28fe-5d69-43bd-afad-26658905dba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:31:10.821554 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-df9a28fe-5d69-43bd-afad-26658905dba8 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:31:10.822796 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-df9a28fe-5d69-43bd-afad-26658905dba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:31:10.830654 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1285/2584] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:31:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:31:10.834750 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c222e9e-88d9-43d0-ab62-c1f58adcb3b0 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:31:10.844325 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c222e9e-88d9-43d0-ab62-c1f58adcb3b0 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:31:10.849045 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e02f7d-c66d-4da4-9c8a-2504a35b068a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:31:10.850150 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e02f7d-c66d-4da4-9c8a-2504a35b068a admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:31:10.850476 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e02f7d-c66d-4da4-9c8a-2504a35b068a admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:31:10.850751 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e02f7d-c66d-4da4-9c8a-2504a35b068a admin admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:31:10.851439 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e02f7d-c66d-4da4-9c8a-2504a35b068a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:31:10.851752 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e02f7d-c66d-4da4-9c8a-2504a35b068a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:31:10.853342 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e02f7d-c66d-4da4-9c8a-2504a35b068a admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:31:10.864717 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1300/2585] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Aug 25 03:31:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:31:10.869195 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9a28fe-5d69-43bd-afad-26658905dba8 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:31:10.878478 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9a28fe-5d69-43bd-afad-26658905dba8 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:31:10.883221 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49278eb-c6a4-4658-badb-15895aba2de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:31:10.883817 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49278eb-c6a4-4658-badb-15895aba2de8 admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:31:10.884099 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49278eb-c6a4-4658-badb-15895aba2de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:31:10.884376 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49278eb-c6a4-4658-badb-15895aba2de8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:31:10.884993 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49278eb-c6a4-4658-badb-15895aba2de8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:31:10.886934 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49278eb-c6a4-4658-badb-15895aba2de8 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:31:10.947286 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1286/2586] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:31:10 2022] POST /identity/v3/projects => generated 397 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:31:10.951481 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e02f7d-c66d-4da4-9c8a-2504a35b068a admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:31:10.961563 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e02f7d-c66d-4da4-9c8a-2504a35b068a admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:31:11.016779 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99510c6b-23e9-4985-b62e-42b58cf93d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:31:11.017380 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99510c6b-23e9-4985-b62e-42b58cf93d7b admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:31:11.017662 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99510c6b-23e9-4985-b62e-42b58cf93d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:31:11.017944 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99510c6b-23e9-4985-b62e-42b58cf93d7b admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:31:11.018614 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99510c6b-23e9-4985-b62e-42b58cf93d7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:31:11.020573 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99510c6b-23e9-4985-b62e-42b58cf93d7b admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:31:11.089484 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1301/2587] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:31:10 2022] POST /identity/v3/users => generated 299 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:31:11.093555 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49278eb-c6a4-4658-badb-15895aba2de8 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:31:11.099181 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:31:11.105365 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49278eb-c6a4-4658-badb-15895aba2de8 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:31:11.110419 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c70820-b320-4f3f-8d8b-80e2a0908de9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:31:11.111064 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c70820-b320-4f3f-8d8b-80e2a0908de9 admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:31:11.111346 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c70820-b320-4f3f-8d8b-80e2a0908de9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:31:11.111623 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c70820-b320-4f3f-8d8b-80e2a0908de9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:31:11.112272 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c70820-b320-4f3f-8d8b-80e2a0908de9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:31:11.113870 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c70820-b320-4f3f-8d8b-80e2a0908de9 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:31:11.123976 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1287/2588] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:31:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:31:11.128225 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99510c6b-23e9-4985-b62e-42b58cf93d7b admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:31:11.138347 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99510c6b-23e9-4985-b62e-42b58cf93d7b admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:31:11.143341 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c2dfd3-382b-4445-b1ac-390454dcebc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:31:11.143957 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c2dfd3-382b-4445-b1ac-390454dcebc7 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:31:11.144237 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c2dfd3-382b-4445-b1ac-390454dcebc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:31:11.144513 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c2dfd3-382b-4445-b1ac-390454dcebc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:31:11.145179 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c2dfd3-382b-4445-b1ac-390454dcebc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:31:11.147301 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c2dfd3-382b-4445-b1ac-390454dcebc7 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:31:11.158008 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1302/2589] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:31:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:31:11.162857 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c70820-b320-4f3f-8d8b-80e2a0908de9 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:31:11.173264 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c70820-b320-4f3f-8d8b-80e2a0908de9 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:31:11.178074 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40225ae9-b659-470c-9d32-f4c4c277b639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:31:11.180437 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40225ae9-b659-470c-9d32-f4c4c277b639 admin admin] REQUEST_METHOD: `PUT` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:31:11.180437 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40225ae9-b659-470c-9d32-f4c4c277b639 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:31:11.180437 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40225ae9-b659-470c-9d32-f4c4c277b639 admin admin] PATH_INFO: `/v3/projects/09fb5940c2eb44cdbfa1ac855351ac1c/users/9b20c99399c540ed878dd9a52ffb7942/roles/3255f77192524a16bc405173106871fc` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:31:11.194461 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40225ae9-b659-470c-9d32-f4c4c277b639 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09fb5940c2eb44cdbfa1ac855351ac1c, user_id=9b20c99399c540ed878dd9a52ffb7942, role_id=3255f77192524a16bc405173106871fc)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:31:11.197122 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40225ae9-b659-470c-9d32-f4c4c277b639 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:31:11.260324 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1288/2590] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:31:11 2022] PUT /identity/v3/projects/09fb5940c2eb44cdbfa1ac855351ac1c/users/9b20c99399c540ed878dd9a52ffb7942/roles/3255f77192524a16bc405173106871fc => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:31:11.270785 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b189fd4-9316-440b-ba83-d003465efdeb None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:31:11.271572 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b189fd4-9316-440b-ba83-d003465efdeb None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:31:11.271861 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b189fd4-9316-440b-ba83-d003465efdeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:31:11.299695 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-4b189fd4-9316-440b-ba83-d003465efdeb None None] MFA Rules not processed for user `9b20c99399c540ed878dd9a52ffb7942`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:31:11.300909 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-4b189fd4-9316-440b-ba83-d003465efdeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:31:11.347944 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:31:11.364537 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1303/2591] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:31:11 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:31:11.377234 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40225ae9-b659-470c-9d32-f4c4c277b639 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:31:11.378966 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-40225ae9-b659-470c-9d32-f4c4c277b639 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:31:11.435767 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40225ae9-b659-470c-9d32-f4c4c277b639 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:31:11.439394 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5db0620-cf5f-4688-af75-c76132bc4980 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60193ba9e1c14df3bc9eebcc6f28f2d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:31:11.439999 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5db0620-cf5f-4688-af75-c76132bc4980 service cinder] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:31:11.440280 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5db0620-cf5f-4688-af75-c76132bc4980 service cinder] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:31:11.440556 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5db0620-cf5f-4688-af75-c76132bc4980 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:31:11.449294 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5db0620-cf5f-4688-af75-c76132bc4980 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:31:11.450753 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5db0620-cf5f-4688-af75-c76132bc4980 service cinder] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:31:11.459994 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1289/2592] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:31:11 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:31:21.420317 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=4 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:31:21.455464 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=40,keystone:INSERT=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:31:23.502691 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b189fd4-9316-440b-ba83-d003465efdeb None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:31:23.564435 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b189fd4-9316-440b-ba83-d003465efdeb None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:31:23.568914 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807dc272-35a3-44cf-bc4a-0b58f7e8f70f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:31:23.569547 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807dc272-35a3-44cf-bc4a-0b58f7e8f70f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:31:23.569828 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807dc272-35a3-44cf-bc4a-0b58f7e8f70f admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:31:23.570104 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807dc272-35a3-44cf-bc4a-0b58f7e8f70f admin admin] PATH_INFO: `/v3/users/9b20c99399c540ed878dd9a52ffb7942` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:31:23.571323 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807dc272-35a3-44cf-bc4a-0b58f7e8f70f admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b20c99399c540ed878dd9a52ffb7942)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:31:23.573248 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807dc272-35a3-44cf-bc4a-0b58f7e8f70f admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:31:23.759958 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-807dc272-35a3-44cf-bc4a-0b58f7e8f70f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b20c99399c540ed878dd9a52ffb7942'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:31:23.825683 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-807dc272-35a3-44cf-bc4a-0b58f7e8f70f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b20c99399c540ed878dd9a52ffb7942'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:31:23.826330 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-807dc272-35a3-44cf-bc4a-0b58f7e8f70f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b20c99399c540ed878dd9a52ffb7942'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:31:23.833477 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-807dc272-35a3-44cf-bc4a-0b58f7e8f70f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b20c99399c540ed878dd9a52ffb7942'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:31:23.845541 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-807dc272-35a3-44cf-bc4a-0b58f7e8f70f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b20c99399c540ed878dd9a52ffb7942'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:31:23.852048 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1304/2593] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:31:23 2022] DELETE /identity/v3/users/9b20c99399c540ed878dd9a52ffb7942 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:31:23.856742 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a9a414-710d-41bb-8def-c68f46b3cbaa None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:31:23.857217 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a9a414-710d-41bb-8def-c68f46b3cbaa None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:31:23.857343 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a9a414-710d-41bb-8def-c68f46b3cbaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:31:23.875652 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-a8a9a414-710d-41bb-8def-c68f46b3cbaa None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:31:23.876846 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-a8a9a414-710d-41bb-8def-c68f46b3cbaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:31:23.932839 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1290/2594] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:31:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:31:23.942890 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807dc272-35a3-44cf-bc4a-0b58f7e8f70f admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:31:23.944490 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-807dc272-35a3-44cf-bc4a-0b58f7e8f70f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:31:24.000367 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807dc272-35a3-44cf-bc4a-0b58f7e8f70f admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:31:24.003977 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf53f745-ac60-446e-b0e8-30b3d32024e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:31:24.004526 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf53f745-ac60-446e-b0e8-30b3d32024e9 service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:31:24.004809 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf53f745-ac60-446e-b0e8-30b3d32024e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:31:24.005078 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf53f745-ac60-446e-b0e8-30b3d32024e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:31:24.013710 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf53f745-ac60-446e-b0e8-30b3d32024e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:31:24.015154 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf53f745-ac60-446e-b0e8-30b3d32024e9 service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:31:24.024814 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1305/2595] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:31:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:31:24.416258 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a9a414-710d-41bb-8def-c68f46b3cbaa None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:31:24.417702 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-a8a9a414-710d-41bb-8def-c68f46b3cbaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:31:24.429767 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a9a414-710d-41bb-8def-c68f46b3cbaa None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:31:24.434758 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed13e5b9-7a06-4ff9-8bdd-57420a6d7ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:31:24.435427 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed13e5b9-7a06-4ff9-8bdd-57420a6d7ccc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:31:24.435710 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed13e5b9-7a06-4ff9-8bdd-57420a6d7ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:31:24.435986 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed13e5b9-7a06-4ff9-8bdd-57420a6d7ccc admin admin] PATH_INFO: `/v3/projects/09fb5940c2eb44cdbfa1ac855351ac1c` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:31:24.437103 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed13e5b9-7a06-4ff9-8bdd-57420a6d7ccc admin admin] RBAC: Authorizing `identity:delete_project(project_id=09fb5940c2eb44cdbfa1ac855351ac1c)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:31:24.439239 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed13e5b9-7a06-4ff9-8bdd-57420a6d7ccc admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:31:24.545830 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-ed13e5b9-7a06-4ff9-8bdd-57420a6d7ccc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09fb5940c2eb44cdbfa1ac855351ac1c', 'request_id': 'req-ed13e5b9-7a06-4ff9-8bdd-57420a6d7ccc'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:31:24.638042 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-ed13e5b9-7a06-4ff9-8bdd-57420a6d7ccc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09fb5940c2eb44cdbfa1ac855351ac1c', 'request_id': 'req-ed13e5b9-7a06-4ff9-8bdd-57420a6d7ccc'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:31:24.643953 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-ed13e5b9-7a06-4ff9-8bdd-57420a6d7ccc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09fb5940c2eb44cdbfa1ac855351ac1c', 'request_id': 'req-ed13e5b9-7a06-4ff9-8bdd-57420a6d7ccc'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:31:24.655115 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-ed13e5b9-7a06-4ff9-8bdd-57420a6d7ccc admin admin] The token cache is being invalidate because project 09fb5940c2eb44cdbfa1ac855351ac1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81995) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:31:24.655405 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-ed13e5b9-7a06-4ff9-8bdd-57420a6d7ccc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:31:24.656038 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-ed13e5b9-7a06-4ff9-8bdd-57420a6d7ccc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:31:24.657517 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1291/2596] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:31:24 2022] DELETE /identity/v3/projects/09fb5940c2eb44cdbfa1ac855351ac1c => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:31:34.020317 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:31:34.652096 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:31:48.584791 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf53f745-ac60-446e-b0e8-30b3d32024e9 service neutron] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:31:48.586188 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-bf53f745-ac60-446e-b0e8-30b3d32024e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:31:48.644125 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf53f745-ac60-446e-b0e8-30b3d32024e9 service neutron] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:31:48.692202 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b9418f-557c-438e-8ae8-2de440d109d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60193ba9e1c14df3bc9eebcc6f28f2d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:31:48.692859 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b9418f-557c-438e-8ae8-2de440d109d1 service cinder] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:31:48.693142 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b9418f-557c-438e-8ae8-2de440d109d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:31:48.693419 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b9418f-557c-438e-8ae8-2de440d109d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:31:48.695231 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-69b9418f-557c-438e-8ae8-2de440d109d1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:31:48.740090 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed13e5b9-7a06-4ff9-8bdd-57420a6d7ccc admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:31:48.742118 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-ed13e5b9-7a06-4ff9-8bdd-57420a6d7ccc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:31:48.755767 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed13e5b9-7a06-4ff9-8bdd-57420a6d7ccc admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:31:48.759455 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf02c756-b7d7-4cc5-9077-af4b6d8c7250 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60193ba9e1c14df3bc9eebcc6f28f2d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:31:48.760057 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf02c756-b7d7-4cc5-9077-af4b6d8c7250 service cinder] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:31:48.760933 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf02c756-b7d7-4cc5-9077-af4b6d8c7250 service cinder] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:31:48.761204 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf02c756-b7d7-4cc5-9077-af4b6d8c7250 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:31:48.763085 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-bf02c756-b7d7-4cc5-9077-af4b6d8c7250 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:31:48.777139 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b9418f-557c-438e-8ae8-2de440d109d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:31:48.778612 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b9418f-557c-438e-8ae8-2de440d109d1 service cinder] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:31:48.797328 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1306/2597] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:31:48 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:31:48.825323 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf02c756-b7d7-4cc5-9077-af4b6d8c7250 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:31:48.826862 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf02c756-b7d7-4cc5-9077-af4b6d8c7250 service cinder] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:31:48.836133 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1292/2598] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:31:48 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 25 03:31:58.794822 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=81 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:31:58.831602 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=28 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:32:03.797788 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b9418f-557c-438e-8ae8-2de440d109d1 service cinder] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:32:03.799320 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-69b9418f-557c-438e-8ae8-2de440d109d1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:32:03.862893 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b9418f-557c-438e-8ae8-2de440d109d1 service cinder] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:32:03.867137 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3ceec5-6e64-4d7e-8b6a-e73b05aa9d7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:32:03.867863 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3ceec5-6e64-4d7e-8b6a-e73b05aa9d7b service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:03.868244 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3ceec5-6e64-4d7e-8b6a-e73b05aa9d7b service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:03.868618 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3ceec5-6e64-4d7e-8b6a-e73b05aa9d7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:03.872037 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-cc3ceec5-6e64-4d7e-8b6a-e73b05aa9d7b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:32:03.951841 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3ceec5-6e64-4d7e-8b6a-e73b05aa9d7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:32:03.953321 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3ceec5-6e64-4d7e-8b6a-e73b05aa9d7b service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:32:03.964036 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1307/2599] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:32:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:32:07.472233 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf02c756-b7d7-4cc5-9077-af4b6d8c7250 service cinder] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:32:07.477062 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-bf02c756-b7d7-4cc5-9077-af4b6d8c7250 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:32:07.491333 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf02c756-b7d7-4cc5-9077-af4b6d8c7250 service cinder] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:32:07.496339 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b03062e-75cf-42c0-b54d-9acaee079b93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:32:07.496973 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b03062e-75cf-42c0-b54d-9acaee079b93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:07.497248 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b03062e-75cf-42c0-b54d-9acaee079b93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:07.497550 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b03062e-75cf-42c0-b54d-9acaee079b93 admin admin] PATH_INFO: `/v3/users/68b69b25b19a4addb9aff7fb7a710e56` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:07.498707 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b03062e-75cf-42c0-b54d-9acaee079b93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=68b69b25b19a4addb9aff7fb7a710e56)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:32:07.500612 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b03062e-75cf-42c0-b54d-9acaee079b93 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:32:07.655390 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-5b03062e-75cf-42c0-b54d-9acaee079b93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68b69b25b19a4addb9aff7fb7a710e56'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:07.713448 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-5b03062e-75cf-42c0-b54d-9acaee079b93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68b69b25b19a4addb9aff7fb7a710e56'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:07.714719 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-5b03062e-75cf-42c0-b54d-9acaee079b93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68b69b25b19a4addb9aff7fb7a710e56'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:07.722053 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-5b03062e-75cf-42c0-b54d-9acaee079b93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68b69b25b19a4addb9aff7fb7a710e56'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:07.746323 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-5b03062e-75cf-42c0-b54d-9acaee079b93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68b69b25b19a4addb9aff7fb7a710e56'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:07.761532 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1293/2600] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:32:07 2022] DELETE /identity/v3/users/68b69b25b19a4addb9aff7fb7a710e56 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:32:08.097591 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3ceec5-6e64-4d7e-8b6a-e73b05aa9d7b service neutron] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:32:08.121824 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3ceec5-6e64-4d7e-8b6a-e73b05aa9d7b service neutron] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:32:08.180539 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e83cee9-c13c-4469-8375-f37416fb4278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:32:08.184188 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e83cee9-c13c-4469-8375-f37416fb4278 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:08.184188 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e83cee9-c13c-4469-8375-f37416fb4278 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:08.184188 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e83cee9-c13c-4469-8375-f37416fb4278 admin admin] PATH_INFO: `/v3/projects/a48e4c9aca394fc1a3bb497c26f2a9f1` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:08.184188 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e83cee9-c13c-4469-8375-f37416fb4278 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a48e4c9aca394fc1a3bb497c26f2a9f1)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:32:08.185602 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e83cee9-c13c-4469-8375-f37416fb4278 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:32:08.336625 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-9e83cee9-c13c-4469-8375-f37416fb4278 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a48e4c9aca394fc1a3bb497c26f2a9f1', 'request_id': 'req-9e83cee9-c13c-4469-8375-f37416fb4278'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:08.340484 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:32:08.471219 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-9e83cee9-c13c-4469-8375-f37416fb4278 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a48e4c9aca394fc1a3bb497c26f2a9f1', 'request_id': 'req-9e83cee9-c13c-4469-8375-f37416fb4278'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:08.477239 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-9e83cee9-c13c-4469-8375-f37416fb4278 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a48e4c9aca394fc1a3bb497c26f2a9f1', 'request_id': 'req-9e83cee9-c13c-4469-8375-f37416fb4278'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:08.488812 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-9e83cee9-c13c-4469-8375-f37416fb4278 admin admin] The token cache is being invalidate because project a48e4c9aca394fc1a3bb497c26f2a9f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:32:08.489140 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-9e83cee9-c13c-4469-8375-f37416fb4278 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:08.489800 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-9e83cee9-c13c-4469-8375-f37416fb4278 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:08.491414 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1308/2601] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:32:08 2022] DELETE /identity/v3/projects/a48e4c9aca394fc1a3bb497c26f2a9f1 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:32:08.496901 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38735cf4-00ca-4c89-b1e7-5d96e6cd5932 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:08.497394 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38735cf4-00ca-4c89-b1e7-5d96e6cd5932 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:08.497505 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38735cf4-00ca-4c89-b1e7-5d96e6cd5932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:08.517088 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-38735cf4-00ca-4c89-b1e7-5d96e6cd5932 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:32:08.518332 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-38735cf4-00ca-4c89-b1e7-5d96e6cd5932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:32:08.578886 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1294/2602] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:32:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:32:08.587377 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3310a5c-59c7-49c8-864f-3b226838c6b4 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:08.587952 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3310a5c-59c7-49c8-864f-3b226838c6b4 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:08.588556 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3310a5c-59c7-49c8-864f-3b226838c6b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:08.609405 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-b3310a5c-59c7-49c8-864f-3b226838c6b4 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:32:08.618945 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-b3310a5c-59c7-49c8-864f-3b226838c6b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:32:08.618945 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1309/2603] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:32:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:32:08.624540 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38735cf4-00ca-4c89-b1e7-5d96e6cd5932 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:32:08.636494 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38735cf4-00ca-4c89-b1e7-5d96e6cd5932 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:32:08.642404 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd53ee3-5318-4963-8c2b-84186678e55d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:32:08.647556 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd53ee3-5318-4963-8c2b-84186678e55d admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:08.647556 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd53ee3-5318-4963-8c2b-84186678e55d admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:08.647556 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd53ee3-5318-4963-8c2b-84186678e55d admin admin] PATH_INFO: `/v3/domains` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:08.647556 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd53ee3-5318-4963-8c2b-84186678e55d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:32:08.647556 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd53ee3-5318-4963-8c2b-84186678e55d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:32:08.647556 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd53ee3-5318-4963-8c2b-84186678e55d admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:32:08.661335 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1295/2604] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Aug 25 03:32:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:32:08.665956 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3310a5c-59c7-49c8-864f-3b226838c6b4 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:32:08.675945 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3310a5c-59c7-49c8-864f-3b226838c6b4 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:32:08.681564 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e102dad5-a3b8-4048-8e32-560a074ecb03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:32:08.682108 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e102dad5-a3b8-4048-8e32-560a074ecb03 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:08.682433 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e102dad5-a3b8-4048-8e32-560a074ecb03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:08.682786 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e102dad5-a3b8-4048-8e32-560a074ecb03 admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:08.683691 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e102dad5-a3b8-4048-8e32-560a074ecb03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:32:08.685592 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e102dad5-a3b8-4048-8e32-560a074ecb03 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:32:08.838604 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1310/2605] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:32:08 2022] POST /identity/v3/projects => generated 377 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:32:08.843562 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd53ee3-5318-4963-8c2b-84186678e55d admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:32:08.853717 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd53ee3-5318-4963-8c2b-84186678e55d admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:32:08.911943 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4436d2c-b7aa-4d5f-9ae6-1ad4b440e484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:32:08.912548 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4436d2c-b7aa-4d5f-9ae6-1ad4b440e484 admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:08.912828 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4436d2c-b7aa-4d5f-9ae6-1ad4b440e484 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:08.913100 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4436d2c-b7aa-4d5f-9ae6-1ad4b440e484 admin admin] PATH_INFO: `/v3/users` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:08.913744 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4436d2c-b7aa-4d5f-9ae6-1ad4b440e484 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:32:08.915750 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4436d2c-b7aa-4d5f-9ae6-1ad4b440e484 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:32:08.924436 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:32:08.999356 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1296/2606] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:32:08 2022] POST /identity/v3/users => generated 289 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:32:09.003848 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e102dad5-a3b8-4048-8e32-560a074ecb03 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:32:09.016714 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e102dad5-a3b8-4048-8e32-560a074ecb03 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:32:09.021519 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb311668-6985-4f35-8fdb-00968e9c9ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:32:09.026020 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb311668-6985-4f35-8fdb-00968e9c9ee5 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:09.026020 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb311668-6985-4f35-8fdb-00968e9c9ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:09.026020 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb311668-6985-4f35-8fdb-00968e9c9ee5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:09.026020 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb311668-6985-4f35-8fdb-00968e9c9ee5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:32:09.026020 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb311668-6985-4f35-8fdb-00968e9c9ee5 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:32:09.037150 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1311/2607] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:32:09 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:32:09.041619 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4436d2c-b7aa-4d5f-9ae6-1ad4b440e484 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:32:09.051925 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4436d2c-b7aa-4d5f-9ae6-1ad4b440e484 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:32:09.056639 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952df465-41a8-4a85-811f-996bfccdc5ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:32:09.057233 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952df465-41a8-4a85-811f-996bfccdc5ba admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:09.057511 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952df465-41a8-4a85-811f-996bfccdc5ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:09.057783 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952df465-41a8-4a85-811f-996bfccdc5ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:09.058747 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952df465-41a8-4a85-811f-996bfccdc5ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:32:09.060329 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952df465-41a8-4a85-811f-996bfccdc5ba admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:32:09.071065 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1297/2608] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:32:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:32:09.076000 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb311668-6985-4f35-8fdb-00968e9c9ee5 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:32:09.086063 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb311668-6985-4f35-8fdb-00968e9c9ee5 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:32:09.091482 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febb40f6-aacf-41f7-802c-130fe5195e4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:32:09.092132 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febb40f6-aacf-41f7-802c-130fe5195e4d admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:09.092411 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febb40f6-aacf-41f7-802c-130fe5195e4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:09.092759 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febb40f6-aacf-41f7-802c-130fe5195e4d admin admin] PATH_INFO: `/v3/projects/164d99e16ea74ab88185d9ce8deb0f9c/users/c756d30b3a6142d5a9f680c094a939bb/roles/3255f77192524a16bc405173106871fc` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:09.108905 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febb40f6-aacf-41f7-802c-130fe5195e4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=164d99e16ea74ab88185d9ce8deb0f9c, user_id=c756d30b3a6142d5a9f680c094a939bb, role_id=3255f77192524a16bc405173106871fc)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:32:09.111587 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febb40f6-aacf-41f7-802c-130fe5195e4d admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:32:09.151432 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1312/2609] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:32:09 2022] PUT /identity/v3/projects/164d99e16ea74ab88185d9ce8deb0f9c/users/c756d30b3a6142d5a9f680c094a939bb/roles/3255f77192524a16bc405173106871fc => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:32:09.165380 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1805b5-604b-43af-a996-af1d28131f8d None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:09.165669 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1805b5-604b-43af-a996-af1d28131f8d None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:09.165948 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1805b5-604b-43af-a996-af1d28131f8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:09.193763 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-fb1805b5-604b-43af-a996-af1d28131f8d None None] MFA Rules not processed for user `c756d30b3a6142d5a9f680c094a939bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:32:09.195057 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-fb1805b5-604b-43af-a996-af1d28131f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:32:09.258884 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1298/2610] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:32:09 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:32:09.263851 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febb40f6-aacf-41f7-802c-130fe5195e4d admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:32:09.329007 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febb40f6-aacf-41f7-802c-130fe5195e4d admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:32:09.333628 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2e7511-a6e7-45b2-ad1c-adc6d8a64729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:32:09.334232 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2e7511-a6e7-45b2-ad1c-adc6d8a64729 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:09.334619 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2e7511-a6e7-45b2-ad1c-adc6d8a64729 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:09.334895 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2e7511-a6e7-45b2-ad1c-adc6d8a64729 admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:09.335530 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2e7511-a6e7-45b2-ad1c-adc6d8a64729 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:32:09.337497 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2e7511-a6e7-45b2-ad1c-adc6d8a64729 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:32:09.428403 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1313/2611] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:32:09 2022] POST /identity/v3/projects => generated 379 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:32:09.432338 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1805b5-604b-43af-a996-af1d28131f8d None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:32:09.445655 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1805b5-604b-43af-a996-af1d28131f8d None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:32:09.509409 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc6848f-eff3-403d-8564-06afd2a57b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:32:09.510124 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc6848f-eff3-403d-8564-06afd2a57b57 admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:09.510303 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc6848f-eff3-403d-8564-06afd2a57b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:09.510612 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc6848f-eff3-403d-8564-06afd2a57b57 admin admin] PATH_INFO: `/v3/users` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:09.511261 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc6848f-eff3-403d-8564-06afd2a57b57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:32:09.514462 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc6848f-eff3-403d-8564-06afd2a57b57 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:32:09.592160 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1299/2612] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:32:09 2022] POST /identity/v3/users => generated 290 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:32:09.596448 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2e7511-a6e7-45b2-ad1c-adc6d8a64729 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:32:09.606527 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2e7511-a6e7-45b2-ad1c-adc6d8a64729 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:32:09.611037 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3e1050-b8d6-4ebe-a4cd-08c5ffb31ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:32:09.611633 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3e1050-b8d6-4ebe-a4cd-08c5ffb31ca3 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:09.611913 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3e1050-b8d6-4ebe-a4cd-08c5ffb31ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:09.612185 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3e1050-b8d6-4ebe-a4cd-08c5ffb31ca3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:09.612827 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3e1050-b8d6-4ebe-a4cd-08c5ffb31ca3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:32:09.614470 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3e1050-b8d6-4ebe-a4cd-08c5ffb31ca3 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:32:09.625415 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1314/2613] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:32:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:32:09.629281 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc6848f-eff3-403d-8564-06afd2a57b57 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:32:09.642766 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc6848f-eff3-403d-8564-06afd2a57b57 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:32:09.647596 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c09da2-480c-4e2a-93b4-9a9f27686926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:32:09.648202 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c09da2-480c-4e2a-93b4-9a9f27686926 admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:09.648480 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c09da2-480c-4e2a-93b4-9a9f27686926 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:09.649320 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c09da2-480c-4e2a-93b4-9a9f27686926 admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:09.649432 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c09da2-480c-4e2a-93b4-9a9f27686926 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:32:09.651066 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c09da2-480c-4e2a-93b4-9a9f27686926 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:32:09.661246 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1300/2614] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:32:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:32:09.665654 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3e1050-b8d6-4ebe-a4cd-08c5ffb31ca3 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:32:09.679543 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3e1050-b8d6-4ebe-a4cd-08c5ffb31ca3 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:32:09.681613 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94a568f-bfe2-4290-a66d-e12851793cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:32:09.682383 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94a568f-bfe2-4290-a66d-e12851793cf5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:09.682665 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94a568f-bfe2-4290-a66d-e12851793cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:09.682937 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94a568f-bfe2-4290-a66d-e12851793cf5 admin admin] PATH_INFO: `/v3/projects/c5d186c252374f0e8c7e442821b493dc/users/65937f78e7184c8d827f066a6e598760/roles/3255f77192524a16bc405173106871fc` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:09.717175 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94a568f-bfe2-4290-a66d-e12851793cf5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5d186c252374f0e8c7e442821b493dc, user_id=65937f78e7184c8d827f066a6e598760, role_id=3255f77192524a16bc405173106871fc)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:32:09.720755 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94a568f-bfe2-4290-a66d-e12851793cf5 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:32:09.785599 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c09da2-480c-4e2a-93b4-9a9f27686926 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:32:09.789899 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-50c09da2-480c-4e2a-93b4-9a9f27686926 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:32:09.792159 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1315/2615] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:32:09 2022] PUT /identity/v3/projects/c5d186c252374f0e8c7e442821b493dc/users/65937f78e7184c8d827f066a6e598760/roles/3255f77192524a16bc405173106871fc => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:32:09.803391 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c61c79-a8b7-4489-80d5-793d9489869a None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:09.803681 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c61c79-a8b7-4489-80d5-793d9489869a None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:09.803961 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c61c79-a8b7-4489-80d5-793d9489869a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:09.841624 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-51c61c79-a8b7-4489-80d5-793d9489869a None None] MFA Rules not processed for user `65937f78e7184c8d827f066a6e598760`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:32:09.843147 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-51c61c79-a8b7-4489-80d5-793d9489869a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:32:09.847801 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:32:09.853594 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c09da2-480c-4e2a-93b4-9a9f27686926 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:32:09.861563 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:32:09.912252 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf0e8ea-0e2b-4386-bbdd-8e7bf6108a6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:32:09.912877 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf0e8ea-0e2b-4386-bbdd-8e7bf6108a6a service neutron] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:09.913254 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf0e8ea-0e2b-4386-bbdd-8e7bf6108a6a service neutron] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:09.913540 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf0e8ea-0e2b-4386-bbdd-8e7bf6108a6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:09.915450 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-eaf0e8ea-0e2b-4386-bbdd-8e7bf6108a6a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:32:09.919937 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1316/2616] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:32:09 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:32:09.924528 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c61c79-a8b7-4489-80d5-793d9489869a None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:32:09.992498 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf0e8ea-0e2b-4386-bbdd-8e7bf6108a6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:32:09.994133 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf0e8ea-0e2b-4386-bbdd-8e7bf6108a6a service neutron] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:32:09.995500 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c61c79-a8b7-4489-80d5-793d9489869a None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:32:10.001785 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c74cb9-62ae-4c42-a35f-f13c57af5653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:32:10.003360 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c74cb9-62ae-4c42-a35f-f13c57af5653 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:10.004217 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c74cb9-62ae-4c42-a35f-f13c57af5653 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:10.004496 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c74cb9-62ae-4c42-a35f-f13c57af5653 admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:10.005136 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c74cb9-62ae-4c42-a35f-f13c57af5653 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:32:10.006552 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1301/2617] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:32:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:32:10.007462 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c74cb9-62ae-4c42-a35f-f13c57af5653 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:32:10.078194 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1317/2618] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:32:09 2022] POST /identity/v3/projects => generated 379 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:32:10.082267 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf0e8ea-0e2b-4386-bbdd-8e7bf6108a6a service neutron] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:32:10.091928 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf0e8ea-0e2b-4386-bbdd-8e7bf6108a6a service neutron] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:32:10.150741 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad39421-5fd3-4c73-87e6-d9e8d899f817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:32:10.151382 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad39421-5fd3-4c73-87e6-d9e8d899f817 admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:10.151682 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad39421-5fd3-4c73-87e6-d9e8d899f817 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:10.151959 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad39421-5fd3-4c73-87e6-d9e8d899f817 admin admin] PATH_INFO: `/v3/users` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:10.153089 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad39421-5fd3-4c73-87e6-d9e8d899f817 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:32:10.155019 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad39421-5fd3-4c73-87e6-d9e8d899f817 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:32:10.249089 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1302/2619] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:32:10 2022] POST /identity/v3/users => generated 296 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:32:10.253072 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c74cb9-62ae-4c42-a35f-f13c57af5653 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:32:10.263743 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c74cb9-62ae-4c42-a35f-f13c57af5653 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:32:10.268344 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5116be0a-d0fd-4210-8b3d-2c85f83a9316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:32:10.269114 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5116be0a-d0fd-4210-8b3d-2c85f83a9316 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:10.269390 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5116be0a-d0fd-4210-8b3d-2c85f83a9316 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:10.269657 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5116be0a-d0fd-4210-8b3d-2c85f83a9316 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:10.270399 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5116be0a-d0fd-4210-8b3d-2c85f83a9316 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:32:10.271996 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5116be0a-d0fd-4210-8b3d-2c85f83a9316 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:32:10.283525 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1318/2620] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:32:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:32:10.287675 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad39421-5fd3-4c73-87e6-d9e8d899f817 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:32:10.298638 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad39421-5fd3-4c73-87e6-d9e8d899f817 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:32:10.303337 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cf71e0-722d-4fbd-947e-58f05677ec45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:32:10.303993 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cf71e0-722d-4fbd-947e-58f05677ec45 admin admin] REQUEST_METHOD: `PUT` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:10.304271 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cf71e0-722d-4fbd-947e-58f05677ec45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:10.304546 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cf71e0-722d-4fbd-947e-58f05677ec45 admin admin] PATH_INFO: `/v3/projects/73a800f2dae8407088a34fae17cb52ea/users/f06f7661484b4fc2b53e4756810a4064/roles/62f19ea272b447ccaab1576cd9bf4fd9` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:10.319937 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cf71e0-722d-4fbd-947e-58f05677ec45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73a800f2dae8407088a34fae17cb52ea, user_id=f06f7661484b4fc2b53e4756810a4064, role_id=62f19ea272b447ccaab1576cd9bf4fd9)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:32:10.322801 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cf71e0-722d-4fbd-947e-58f05677ec45 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:32:10.397188 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1303/2621] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:32:10 2022] PUT /identity/v3/projects/73a800f2dae8407088a34fae17cb52ea/users/f06f7661484b4fc2b53e4756810a4064/roles/62f19ea272b447ccaab1576cd9bf4fd9 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:32:10.401146 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5116be0a-d0fd-4210-8b3d-2c85f83a9316 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:32:10.412097 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5116be0a-d0fd-4210-8b3d-2c85f83a9316 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:32:10.473332 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8aec8fb-c704-4afa-8828-b150d355a681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:32:10.473958 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aec8fb-c704-4afa-8828-b150d355a681 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:10.474272 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aec8fb-c704-4afa-8828-b150d355a681 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:10.474577 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aec8fb-c704-4afa-8828-b150d355a681 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:10.475232 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8aec8fb-c704-4afa-8828-b150d355a681 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:32:10.476863 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8aec8fb-c704-4afa-8828-b150d355a681 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:32:10.491565 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1319/2622] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:32:10 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:32:10.493074 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cf71e0-722d-4fbd-947e-58f05677ec45 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:32:10.504103 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cf71e0-722d-4fbd-947e-58f05677ec45 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:32:10.508976 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2673f01-cb73-455d-85c1-fa6f1b7167b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:32:10.509723 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2673f01-cb73-455d-85c1-fa6f1b7167b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:10.510340 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2673f01-cb73-455d-85c1-fa6f1b7167b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:10.510340 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2673f01-cb73-455d-85c1-fa6f1b7167b6 admin admin] PATH_INFO: `/v3/projects/73a800f2dae8407088a34fae17cb52ea/users/f06f7661484b4fc2b53e4756810a4064/roles/62f19ea272b447ccaab1576cd9bf4fd9` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:10.513042 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2673f01-cb73-455d-85c1-fa6f1b7167b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73a800f2dae8407088a34fae17cb52ea, user_id=f06f7661484b4fc2b53e4756810a4064, role_id=62f19ea272b447ccaab1576cd9bf4fd9)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:32:10.515815 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2673f01-cb73-455d-85c1-fa6f1b7167b6 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:32:10.527882 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1304/2623] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:32:10 2022] PUT /identity/v3/projects/73a800f2dae8407088a34fae17cb52ea/users/f06f7661484b4fc2b53e4756810a4064/roles/62f19ea272b447ccaab1576cd9bf4fd9 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:32:10.539850 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5acddfa-f812-4f74-828b-00a685674a3c None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:10.540258 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5acddfa-f812-4f74-828b-00a685674a3c None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:10.540602 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5acddfa-f812-4f74-828b-00a685674a3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:10.570847 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-e5acddfa-f812-4f74-828b-00a685674a3c None None] MFA Rules not processed for user `f06f7661484b4fc2b53e4756810a4064`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:32:10.572083 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-e5acddfa-f812-4f74-828b-00a685674a3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:32:10.592075 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:32:10.650115 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1320/2624] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:32:10 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:32:10.662190 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2673f01-cb73-455d-85c1-fa6f1b7167b6 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:32:10.664550 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-c2673f01-cb73-455d-85c1-fa6f1b7167b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:32:10.670943 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:32:10.722888 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2673f01-cb73-455d-85c1-fa6f1b7167b6 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:32:10.726571 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc660584-03f2-4bb4-b8df-8045f2b775e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60193ba9e1c14df3bc9eebcc6f28f2d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:32:10.727179 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc660584-03f2-4bb4-b8df-8045f2b775e2 service cinder] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:10.727490 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc660584-03f2-4bb4-b8df-8045f2b775e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:10.727732 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc660584-03f2-4bb4-b8df-8045f2b775e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:10.737526 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc660584-03f2-4bb4-b8df-8045f2b775e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:32:10.739211 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc660584-03f2-4bb4-b8df-8045f2b775e2 service cinder] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:32:10.750515 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1305/2625] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:32:10 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:32:10.834351 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5acddfa-f812-4f74-828b-00a685674a3c None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:32:10.835763 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-e5acddfa-f812-4f74-828b-00a685674a3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:32:10.848480 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5acddfa-f812-4f74-828b-00a685674a3c None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:32:10.852199 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88448d0a-3ac5-43bc-950f-d7233cb39db7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60193ba9e1c14df3bc9eebcc6f28f2d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:32:10.852802 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88448d0a-3ac5-43bc-950f-d7233cb39db7 service cinder] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:10.853080 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88448d0a-3ac5-43bc-950f-d7233cb39db7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:10.853368 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88448d0a-3ac5-43bc-950f-d7233cb39db7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:10.914685 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88448d0a-3ac5-43bc-950f-d7233cb39db7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:32:10.916113 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88448d0a-3ac5-43bc-950f-d7233cb39db7 service cinder] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:32:10.925252 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1321/2626] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:32:10 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:32:13.308427 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc660584-03f2-4bb4-b8df-8045f2b775e2 service cinder] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:32:13.310224 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-bc660584-03f2-4bb4-b8df-8045f2b775e2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:32:13.413392 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc660584-03f2-4bb4-b8df-8045f2b775e2 service cinder] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:32:13.421031 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f7d4e3-00d7-49b9-b710-24803ccdc014 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b692edd3e8c41289cc42e1afb4a6dd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:32:13.421936 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f7d4e3-00d7-49b9-b710-24803ccdc014 service placement] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:13.422220 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f7d4e3-00d7-49b9-b710-24803ccdc014 service placement] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:13.422524 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f7d4e3-00d7-49b9-b710-24803ccdc014 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:13.425298 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-e0f7d4e3-00d7-49b9-b710-24803ccdc014 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:32:13.443483 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f7d4e3-00d7-49b9-b710-24803ccdc014 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:32:13.446321 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f7d4e3-00d7-49b9-b710-24803ccdc014 service placement] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:32:13.458234 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1306/2627] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:32:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:32:16.207270 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88448d0a-3ac5-43bc-950f-d7233cb39db7 service cinder] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:32:16.302043 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88448d0a-3ac5-43bc-950f-d7233cb39db7 service cinder] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:32:16.306903 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68af0ae-7909-47c0-8de3-c89caccc091f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:32:16.307571 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68af0ae-7909-47c0-8de3-c89caccc091f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:16.309648 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68af0ae-7909-47c0-8de3-c89caccc091f admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:16.309648 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68af0ae-7909-47c0-8de3-c89caccc091f admin admin] PATH_INFO: `/v3/users/c756d30b3a6142d5a9f680c094a939bb` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:16.309648 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68af0ae-7909-47c0-8de3-c89caccc091f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c756d30b3a6142d5a9f680c094a939bb)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:32:16.311945 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68af0ae-7909-47c0-8de3-c89caccc091f admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:32:16.515252 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-b68af0ae-7909-47c0-8de3-c89caccc091f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c756d30b3a6142d5a9f680c094a939bb'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:16.519129 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:32:16.631808 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-b68af0ae-7909-47c0-8de3-c89caccc091f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c756d30b3a6142d5a9f680c094a939bb'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:16.633181 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-b68af0ae-7909-47c0-8de3-c89caccc091f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c756d30b3a6142d5a9f680c094a939bb'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:16.642103 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-b68af0ae-7909-47c0-8de3-c89caccc091f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c756d30b3a6142d5a9f680c094a939bb'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:16.656276 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-b68af0ae-7909-47c0-8de3-c89caccc091f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c756d30b3a6142d5a9f680c094a939bb'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:16.662461 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1322/2628] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:32:16 2022] DELETE /identity/v3/users/c756d30b3a6142d5a9f680c094a939bb => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:32:16.667795 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebd4a0e-b4f7-4de4-b7be-3614b9728f0f None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:16.668268 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebd4a0e-b4f7-4de4-b7be-3614b9728f0f None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:16.668387 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebd4a0e-b4f7-4de4-b7be-3614b9728f0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:16.688592 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-bebd4a0e-b4f7-4de4-b7be-3614b9728f0f None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:32:16.689829 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-bebd4a0e-b4f7-4de4-b7be-3614b9728f0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:32:16.767998 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1307/2629] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:32:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:32:16.778603 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68af0ae-7909-47c0-8de3-c89caccc091f admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:32:16.780196 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-b68af0ae-7909-47c0-8de3-c89caccc091f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:32:16.841389 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68af0ae-7909-47c0-8de3-c89caccc091f admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:32:16.845381 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eeb26c-81dc-48c6-871d-a412177245aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:32:16.845933 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eeb26c-81dc-48c6-871d-a412177245aa service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:16.846224 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eeb26c-81dc-48c6-871d-a412177245aa service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:16.846571 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eeb26c-81dc-48c6-871d-a412177245aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:16.858649 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6eeb26c-81dc-48c6-871d-a412177245aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:32:16.860545 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6eeb26c-81dc-48c6-871d-a412177245aa service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:32:16.872361 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1323/2630] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:32:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:32:17.484215 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebd4a0e-b4f7-4de4-b7be-3614b9728f0f None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:32:17.484215 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-bebd4a0e-b4f7-4de4-b7be-3614b9728f0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:32:17.495240 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebd4a0e-b4f7-4de4-b7be-3614b9728f0f None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:32:17.501329 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af819733-757e-4c8c-aab7-389f325a98e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:32:17.502401 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af819733-757e-4c8c-aab7-389f325a98e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:17.502599 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af819733-757e-4c8c-aab7-389f325a98e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:17.503021 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af819733-757e-4c8c-aab7-389f325a98e5 admin admin] PATH_INFO: `/v3/projects/164d99e16ea74ab88185d9ce8deb0f9c` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:17.506673 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af819733-757e-4c8c-aab7-389f325a98e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=164d99e16ea74ab88185d9ce8deb0f9c)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:32:17.506673 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af819733-757e-4c8c-aab7-389f325a98e5 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:32:17.597813 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:32:17.629601 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-af819733-757e-4c8c-aab7-389f325a98e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '164d99e16ea74ab88185d9ce8deb0f9c', 'request_id': 'req-af819733-757e-4c8c-aab7-389f325a98e5'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:17.689371 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-af819733-757e-4c8c-aab7-389f325a98e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '164d99e16ea74ab88185d9ce8deb0f9c', 'request_id': 'req-af819733-757e-4c8c-aab7-389f325a98e5'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:17.699868 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-af819733-757e-4c8c-aab7-389f325a98e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '164d99e16ea74ab88185d9ce8deb0f9c', 'request_id': 'req-af819733-757e-4c8c-aab7-389f325a98e5'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:17.732007 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-af819733-757e-4c8c-aab7-389f325a98e5 admin admin] The token cache is being invalidate because project 164d99e16ea74ab88185d9ce8deb0f9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81995) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:32:17.732590 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-af819733-757e-4c8c-aab7-389f325a98e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:17.733790 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-af819733-757e-4c8c-aab7-389f325a98e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:17.735495 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1308/2631] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:32:17 2022] DELETE /identity/v3/projects/164d99e16ea74ab88185d9ce8deb0f9c => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:32:17.739579 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eeb26c-81dc-48c6-871d-a412177245aa service neutron] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:32:17.751778 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eeb26c-81dc-48c6-871d-a412177245aa service neutron] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:32:17.862322 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8c101c-ee40-476f-9b9a-9947182cf667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:32:17.863047 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8c101c-ee40-476f-9b9a-9947182cf667 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:17.864173 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8c101c-ee40-476f-9b9a-9947182cf667 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:17.864602 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8c101c-ee40-476f-9b9a-9947182cf667 admin admin] PATH_INFO: `/v3/users/65937f78e7184c8d827f066a6e598760` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:17.865823 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8c101c-ee40-476f-9b9a-9947182cf667 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65937f78e7184c8d827f066a6e598760)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:32:17.868248 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8c101c-ee40-476f-9b9a-9947182cf667 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:32:18.093153 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-2f8c101c-ee40-476f-9b9a-9947182cf667 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65937f78e7184c8d827f066a6e598760'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:18.156380 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-2f8c101c-ee40-476f-9b9a-9947182cf667 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65937f78e7184c8d827f066a6e598760'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:18.157138 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-2f8c101c-ee40-476f-9b9a-9947182cf667 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65937f78e7184c8d827f066a6e598760'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:18.172692 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-2f8c101c-ee40-476f-9b9a-9947182cf667 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65937f78e7184c8d827f066a6e598760'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:18.202370 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:32:18.222030 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-2f8c101c-ee40-476f-9b9a-9947182cf667 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65937f78e7184c8d827f066a6e598760'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:18.235201 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1324/2632] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:32:17 2022] DELETE /identity/v3/users/65937f78e7184c8d827f066a6e598760 => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:32:18.298230 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af819733-757e-4c8c-aab7-389f325a98e5 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:32:18.299667 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-af819733-757e-4c8c-aab7-389f325a98e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:32:18.377402 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af819733-757e-4c8c-aab7-389f325a98e5 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:32:18.440314 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09103119-d95e-410b-93d6-ffa865fbdc1a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b692edd3e8c41289cc42e1afb4a6dd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:32:18.440927 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09103119-d95e-410b-93d6-ffa865fbdc1a service placement] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:18.441211 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09103119-d95e-410b-93d6-ffa865fbdc1a service placement] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:18.441485 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09103119-d95e-410b-93d6-ffa865fbdc1a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:18.443232 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-09103119-d95e-410b-93d6-ffa865fbdc1a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:32:18.455519 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09103119-d95e-410b-93d6-ffa865fbdc1a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:32:18.457210 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09103119-d95e-410b-93d6-ffa865fbdc1a service placement] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:32:18.469540 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1309/2633] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:32:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:32:18.470510 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c7a7df-b005-4347-880d-003d9fd96f81 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:18.470804 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c7a7df-b005-4347-880d-003d9fd96f81 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:18.471084 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c7a7df-b005-4347-880d-003d9fd96f81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:18.490122 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-d3c7a7df-b005-4347-880d-003d9fd96f81 None None] MFA Rules not processed for user `08a52103c0f4467c8366bd88e8b655e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:32:18.491345 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-d3c7a7df-b005-4347-880d-003d9fd96f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:32:18.518926 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09103119-d95e-410b-93d6-ffa865fbdc1a service placement] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:32:18.520377 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-09103119-d95e-410b-93d6-ffa865fbdc1a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:32:18.570925 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1325/2634] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:32:18 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:32:18.617143 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:32:18.644025 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09103119-d95e-410b-93d6-ffa865fbdc1a service placement] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:32:18.646698 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6821f9b9-c135-4809-bcf4-e4098d4e8af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:32:18.647406 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6821f9b9-c135-4809-bcf4-e4098d4e8af9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:18.648327 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6821f9b9-c135-4809-bcf4-e4098d4e8af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:18.648664 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6821f9b9-c135-4809-bcf4-e4098d4e8af9 admin admin] PATH_INFO: `/v3/users/d0a0e9c327564a3db8b89ea706f0fb36` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:18.665895 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6821f9b9-c135-4809-bcf4-e4098d4e8af9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0a0e9c327564a3db8b89ea706f0fb36)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:32:18.668231 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6821f9b9-c135-4809-bcf4-e4098d4e8af9 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:32:18.794919 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c7a7df-b005-4347-880d-003d9fd96f81 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:32:18.796496 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-d3c7a7df-b005-4347-880d-003d9fd96f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:32:18.817872 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c7a7df-b005-4347-880d-003d9fd96f81 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:32:18.824467 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0edad0-b6eb-4f11-8132-8def4fd37919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:32:18.825106 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0edad0-b6eb-4f11-8132-8def4fd37919 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:18.825410 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-6821f9b9-c135-4809-bcf4-e4098d4e8af9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0a0e9c327564a3db8b89ea706f0fb36'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:18.825926 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0edad0-b6eb-4f11-8132-8def4fd37919 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:18.825926 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0edad0-b6eb-4f11-8132-8def4fd37919 admin admin] PATH_INFO: `/v3/projects/c5d186c252374f0e8c7e442821b493dc` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:18.827754 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0edad0-b6eb-4f11-8132-8def4fd37919 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5d186c252374f0e8c7e442821b493dc)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:32:18.829779 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0edad0-b6eb-4f11-8132-8def4fd37919 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:32:18.888248 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-6821f9b9-c135-4809-bcf4-e4098d4e8af9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0a0e9c327564a3db8b89ea706f0fb36'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:18.888969 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-6821f9b9-c135-4809-bcf4-e4098d4e8af9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0a0e9c327564a3db8b89ea706f0fb36'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:18.889602 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-6821f9b9-c135-4809-bcf4-e4098d4e8af9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0a0e9c327564a3db8b89ea706f0fb36'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:18.902233 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-6821f9b9-c135-4809-bcf4-e4098d4e8af9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0a0e9c327564a3db8b89ea706f0fb36'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:18.920301 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1310/2635] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:32:18 2022] DELETE /identity/v3/users/d0a0e9c327564a3db8b89ea706f0fb36 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:32:18.975594 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-8b0edad0-b6eb-4f11-8132-8def4fd37919 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5d186c252374f0e8c7e442821b493dc', 'request_id': 'req-8b0edad0-b6eb-4f11-8132-8def4fd37919'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:19.048153 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-8b0edad0-b6eb-4f11-8132-8def4fd37919 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5d186c252374f0e8c7e442821b493dc', 'request_id': 'req-8b0edad0-b6eb-4f11-8132-8def4fd37919'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:19.055490 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-8b0edad0-b6eb-4f11-8132-8def4fd37919 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5d186c252374f0e8c7e442821b493dc', 'request_id': 'req-8b0edad0-b6eb-4f11-8132-8def4fd37919'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:19.075406 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-8b0edad0-b6eb-4f11-8132-8def4fd37919 admin admin] The token cache is being invalidate because project c5d186c252374f0e8c7e442821b493dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:32:19.075715 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-8b0edad0-b6eb-4f11-8132-8def4fd37919 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:19.076393 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-8b0edad0-b6eb-4f11-8132-8def4fd37919 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:19.077964 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1326/2636] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:32:18 2022] DELETE /identity/v3/projects/c5d186c252374f0e8c7e442821b493dc => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:32:19.082193 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6821f9b9-c135-4809-bcf4-e4098d4e8af9 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:32:19.083849 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-6821f9b9-c135-4809-bcf4-e4098d4e8af9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:32:19.156656 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0edad0-b6eb-4f11-8132-8def4fd37919 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:32:19.157609 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6821f9b9-c135-4809-bcf4-e4098d4e8af9 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:32:19.161022 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-8b0edad0-b6eb-4f11-8132-8def4fd37919 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:32:19.230022 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243cad3f-1d01-43ed-83f0-42ef8d79d6dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:32:19.231304 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243cad3f-1d01-43ed-83f0-42ef8d79d6dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:19.231590 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243cad3f-1d01-43ed-83f0-42ef8d79d6dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:19.231871 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243cad3f-1d01-43ed-83f0-42ef8d79d6dc admin admin] PATH_INFO: `/v3/users/f06f7661484b4fc2b53e4756810a4064` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:19.233139 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243cad3f-1d01-43ed-83f0-42ef8d79d6dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=f06f7661484b4fc2b53e4756810a4064)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:32:19.235164 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243cad3f-1d01-43ed-83f0-42ef8d79d6dc admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:32:19.241740 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0edad0-b6eb-4f11-8132-8def4fd37919 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:32:19.246581 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca2c4fe-9c40-4fab-ac3c-47631d57375d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:32:19.247317 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca2c4fe-9c40-4fab-ac3c-47631d57375d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:19.247670 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca2c4fe-9c40-4fab-ac3c-47631d57375d admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:19.248017 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca2c4fe-9c40-4fab-ac3c-47631d57375d admin admin] PATH_INFO: `/v3/projects/5d445535a4974186a835ab117109dca1` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:19.259504 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca2c4fe-9c40-4fab-ac3c-47631d57375d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d445535a4974186a835ab117109dca1)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:32:19.261661 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca2c4fe-9c40-4fab-ac3c-47631d57375d admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:32:19.278331 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:32:19.365702 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:32:19.375738 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-8ca2c4fe-9c40-4fab-ac3c-47631d57375d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d445535a4974186a835ab117109dca1', 'request_id': 'req-8ca2c4fe-9c40-4fab-ac3c-47631d57375d'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:19.378502 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-243cad3f-1d01-43ed-83f0-42ef8d79d6dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f06f7661484b4fc2b53e4756810a4064'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:19.442348 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-8ca2c4fe-9c40-4fab-ac3c-47631d57375d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d445535a4974186a835ab117109dca1', 'request_id': 'req-8ca2c4fe-9c40-4fab-ac3c-47631d57375d'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:19.448551 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-8ca2c4fe-9c40-4fab-ac3c-47631d57375d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d445535a4974186a835ab117109dca1', 'request_id': 'req-8ca2c4fe-9c40-4fab-ac3c-47631d57375d'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:19.461458 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-8ca2c4fe-9c40-4fab-ac3c-47631d57375d admin admin] The token cache is being invalidate because project 5d445535a4974186a835ab117109dca1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:32:19.461782 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-8ca2c4fe-9c40-4fab-ac3c-47631d57375d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:19.462516 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-8ca2c4fe-9c40-4fab-ac3c-47631d57375d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:19.465560 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1327/2637] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:32:19 2022] DELETE /identity/v3/projects/5d445535a4974186a835ab117109dca1 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:32:19.468991 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca2c4fe-9c40-4fab-ac3c-47631d57375d admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:32:19.470418 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-8ca2c4fe-9c40-4fab-ac3c-47631d57375d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:32:19.505103 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-243cad3f-1d01-43ed-83f0-42ef8d79d6dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f06f7661484b4fc2b53e4756810a4064'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:19.505724 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-243cad3f-1d01-43ed-83f0-42ef8d79d6dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f06f7661484b4fc2b53e4756810a4064'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:19.506371 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-243cad3f-1d01-43ed-83f0-42ef8d79d6dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f06f7661484b4fc2b53e4756810a4064'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:19.519539 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-243cad3f-1d01-43ed-83f0-42ef8d79d6dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f06f7661484b4fc2b53e4756810a4064'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:19.533332 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1311/2638] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:32:19 2022] DELETE /identity/v3/users/f06f7661484b4fc2b53e4756810a4064 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:32:19.539623 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca2c4fe-9c40-4fab-ac3c-47631d57375d admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:32:19.600942 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5308bce-8d00-4b9e-a6af-31f21453b3a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:32:19.601585 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5308bce-8d00-4b9e-a6af-31f21453b3a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:19.601868 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5308bce-8d00-4b9e-a6af-31f21453b3a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:19.602143 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5308bce-8d00-4b9e-a6af-31f21453b3a3 admin admin] PATH_INFO: `/v3/users/97897f84a40f4401929fe2746945b6e3` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:19.622007 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5308bce-8d00-4b9e-a6af-31f21453b3a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97897f84a40f4401929fe2746945b6e3)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:32:19.622007 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5308bce-8d00-4b9e-a6af-31f21453b3a3 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:32:19.785385 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-e5308bce-8d00-4b9e-a6af-31f21453b3a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97897f84a40f4401929fe2746945b6e3'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:19.791338 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:32:19.882847 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-e5308bce-8d00-4b9e-a6af-31f21453b3a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97897f84a40f4401929fe2746945b6e3'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:19.883434 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-e5308bce-8d00-4b9e-a6af-31f21453b3a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97897f84a40f4401929fe2746945b6e3'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:19.891404 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-e5308bce-8d00-4b9e-a6af-31f21453b3a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97897f84a40f4401929fe2746945b6e3'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:19.903235 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-e5308bce-8d00-4b9e-a6af-31f21453b3a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97897f84a40f4401929fe2746945b6e3'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:19.909370 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1328/2639] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:32:19 2022] DELETE /identity/v3/users/97897f84a40f4401929fe2746945b6e3 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:32:19.975992 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243cad3f-1d01-43ed-83f0-42ef8d79d6dc admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:32:19.978015 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-243cad3f-1d01-43ed-83f0-42ef8d79d6dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:32:19.996675 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243cad3f-1d01-43ed-83f0-42ef8d79d6dc admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:32:20.063281 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7ccf67-8731-40a4-943e-1a85de6b3777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:32:20.063922 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7ccf67-8731-40a4-943e-1a85de6b3777 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:20.064207 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7ccf67-8731-40a4-943e-1a85de6b3777 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:20.064489 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7ccf67-8731-40a4-943e-1a85de6b3777 admin admin] PATH_INFO: `/v3/projects/73a800f2dae8407088a34fae17cb52ea` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:20.065624 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7ccf67-8731-40a4-943e-1a85de6b3777 admin admin] RBAC: Authorizing `identity:delete_project(project_id=73a800f2dae8407088a34fae17cb52ea)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:32:20.067567 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7ccf67-8731-40a4-943e-1a85de6b3777 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:32:20.196495 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5308bce-8d00-4b9e-a6af-31f21453b3a3 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:32:20.196495 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-e5308bce-8d00-4b9e-a6af-31f21453b3a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:32:20.208242 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5308bce-8d00-4b9e-a6af-31f21453b3a3 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:32:20.212762 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e37a98-4681-4bfa-b238-9a9a01e6f475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:32:20.213398 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e37a98-4681-4bfa-b238-9a9a01e6f475 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:20.213678 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e37a98-4681-4bfa-b238-9a9a01e6f475 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:20.213946 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e37a98-4681-4bfa-b238-9a9a01e6f475 admin admin] PATH_INFO: `/v3/projects/3cc897b906b243e395495bfed506497c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:20.225920 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e37a98-4681-4bfa-b238-9a9a01e6f475 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3cc897b906b243e395495bfed506497c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:32:20.227617 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e37a98-4681-4bfa-b238-9a9a01e6f475 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:32:20.261373 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-4e7ccf67-8731-40a4-943e-1a85de6b3777 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73a800f2dae8407088a34fae17cb52ea', 'request_id': 'req-4e7ccf67-8731-40a4-943e-1a85de6b3777'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:20.339430 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-4e7ccf67-8731-40a4-943e-1a85de6b3777 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73a800f2dae8407088a34fae17cb52ea', 'request_id': 'req-4e7ccf67-8731-40a4-943e-1a85de6b3777'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:20.339968 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-d6e37a98-4681-4bfa-b238-9a9a01e6f475 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3cc897b906b243e395495bfed506497c', 'request_id': 'req-d6e37a98-4681-4bfa-b238-9a9a01e6f475'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:20.346616 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-4e7ccf67-8731-40a4-943e-1a85de6b3777 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73a800f2dae8407088a34fae17cb52ea', 'request_id': 'req-4e7ccf67-8731-40a4-943e-1a85de6b3777'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:20.358624 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-4e7ccf67-8731-40a4-943e-1a85de6b3777 admin admin] The token cache is being invalidate because project 73a800f2dae8407088a34fae17cb52ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81995) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:32:20.358919 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-4e7ccf67-8731-40a4-943e-1a85de6b3777 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:20.359573 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-4e7ccf67-8731-40a4-943e-1a85de6b3777 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:20.361089 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1312/2640] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:32:19 2022] DELETE /identity/v3/projects/73a800f2dae8407088a34fae17cb52ea => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:32:20.366741 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04d0a48-466d-4b68-af9b-004212ea0fe1 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:20.367024 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04d0a48-466d-4b68-af9b-004212ea0fe1 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:20.367300 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04d0a48-466d-4b68-af9b-004212ea0fe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:20.388463 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-a04d0a48-466d-4b68-af9b-004212ea0fe1 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:32:20.388824 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-a04d0a48-466d-4b68-af9b-004212ea0fe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:32:20.401255 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:32:20.408069 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-d6e37a98-4681-4bfa-b238-9a9a01e6f475 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3cc897b906b243e395495bfed506497c', 'request_id': 'req-d6e37a98-4681-4bfa-b238-9a9a01e6f475'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:20.415658 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-d6e37a98-4681-4bfa-b238-9a9a01e6f475 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3cc897b906b243e395495bfed506497c', 'request_id': 'req-d6e37a98-4681-4bfa-b238-9a9a01e6f475'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:20.428465 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-d6e37a98-4681-4bfa-b238-9a9a01e6f475 admin admin] The token cache is being invalidate because project 3cc897b906b243e395495bfed506497c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:32:20.428759 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-d6e37a98-4681-4bfa-b238-9a9a01e6f475 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:20.429629 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-d6e37a98-4681-4bfa-b238-9a9a01e6f475 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:20.442398 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1329/2641] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:32:20 2022] DELETE /identity/v3/projects/3cc897b906b243e395495bfed506497c => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:32:20.474888 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1313/2642] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:32:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:32:20.481142 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861239db-d84d-4be2-a2b4-b088dbacbcb1 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:20.481684 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861239db-d84d-4be2-a2b4-b088dbacbcb1 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:20.482210 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861239db-d84d-4be2-a2b4-b088dbacbcb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:20.518585 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-861239db-d84d-4be2-a2b4-b088dbacbcb1 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:32:20.518585 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-861239db-d84d-4be2-a2b4-b088dbacbcb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:32:20.520360 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1330/2643] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:32:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:32:20.524676 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04d0a48-466d-4b68-af9b-004212ea0fe1 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:32:20.535523 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04d0a48-466d-4b68-af9b-004212ea0fe1 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:32:20.540447 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde6cfba-f212-44e6-9a69-bee30be4cb2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:32:20.541078 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde6cfba-f212-44e6-9a69-bee30be4cb2b admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:20.541390 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde6cfba-f212-44e6-9a69-bee30be4cb2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:20.541678 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde6cfba-f212-44e6-9a69-bee30be4cb2b admin admin] PATH_INFO: `/v3/domains` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:20.542382 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde6cfba-f212-44e6-9a69-bee30be4cb2b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:32:20.543234 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde6cfba-f212-44e6-9a69-bee30be4cb2b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:32:20.544951 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde6cfba-f212-44e6-9a69-bee30be4cb2b admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:32:20.562231 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1314/2644] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Aug 25 03:32:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:32:20.563476 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861239db-d84d-4be2-a2b4-b088dbacbcb1 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:32:20.573416 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861239db-d84d-4be2-a2b4-b088dbacbcb1 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:32:20.587308 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4376c21c-8f69-4f83-8a1c-3bbc7ce9d574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:32:20.587949 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4376c21c-8f69-4f83-8a1c-3bbc7ce9d574 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:20.588229 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4376c21c-8f69-4f83-8a1c-3bbc7ce9d574 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:20.588510 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4376c21c-8f69-4f83-8a1c-3bbc7ce9d574 admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:20.589155 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4376c21c-8f69-4f83-8a1c-3bbc7ce9d574 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:32:20.591702 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4376c21c-8f69-4f83-8a1c-3bbc7ce9d574 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:32:20.690236 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1331/2645] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:32:20 2022] POST /identity/v3/projects => generated 387 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:32:20.694361 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde6cfba-f212-44e6-9a69-bee30be4cb2b admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:32:20.704442 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde6cfba-f212-44e6-9a69-bee30be4cb2b admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:32:20.763072 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50519537-d278-43c9-971f-96d88c5bbb6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:32:20.763604 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50519537-d278-43c9-971f-96d88c5bbb6c admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:20.763888 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50519537-d278-43c9-971f-96d88c5bbb6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:20.764167 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50519537-d278-43c9-971f-96d88c5bbb6c admin admin] PATH_INFO: `/v3/users` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:20.764809 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50519537-d278-43c9-971f-96d88c5bbb6c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:32:20.766737 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50519537-d278-43c9-971f-96d88c5bbb6c admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:32:20.850374 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1315/2646] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:32:20 2022] POST /identity/v3/users => generated 294 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:32:20.854751 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4376c21c-8f69-4f83-8a1c-3bbc7ce9d574 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:32:20.866417 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4376c21c-8f69-4f83-8a1c-3bbc7ce9d574 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:32:20.871405 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecd2a30-86a1-48af-98f0-8c1bdb4d771f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:32:20.872543 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecd2a30-86a1-48af-98f0-8c1bdb4d771f admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:20.872543 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecd2a30-86a1-48af-98f0-8c1bdb4d771f admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:20.872798 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecd2a30-86a1-48af-98f0-8c1bdb4d771f admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:20.873423 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecd2a30-86a1-48af-98f0-8c1bdb4d771f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:32:20.875147 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecd2a30-86a1-48af-98f0-8c1bdb4d771f admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:32:20.885635 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1332/2647] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:32:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:32:20.889574 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50519537-d278-43c9-971f-96d88c5bbb6c admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:32:20.899669 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50519537-d278-43c9-971f-96d88c5bbb6c admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:32:20.904961 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848a2d03-5392-4a4d-a774-fd59d91a39f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:32:20.905561 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848a2d03-5392-4a4d-a774-fd59d91a39f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:20.905839 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848a2d03-5392-4a4d-a774-fd59d91a39f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:20.906113 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848a2d03-5392-4a4d-a774-fd59d91a39f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:20.906975 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848a2d03-5392-4a4d-a774-fd59d91a39f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:32:20.908407 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848a2d03-5392-4a4d-a774-fd59d91a39f6 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:32:20.919752 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1316/2648] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:32:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:32:20.923917 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecd2a30-86a1-48af-98f0-8c1bdb4d771f admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:32:20.934460 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecd2a30-86a1-48af-98f0-8c1bdb4d771f admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:32:20.939036 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68d141b-3df7-4899-9bf7-91207f164566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:32:20.941489 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68d141b-3df7-4899-9bf7-91207f164566 admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:20.941489 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68d141b-3df7-4899-9bf7-91207f164566 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:20.941489 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68d141b-3df7-4899-9bf7-91207f164566 admin admin] PATH_INFO: `/v3/projects/cc6164322c9f4826b96da0ac5ed9f3b3/users/2781cd580c12435baee026b6c2e6c725/roles/3255f77192524a16bc405173106871fc` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:20.955139 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68d141b-3df7-4899-9bf7-91207f164566 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc6164322c9f4826b96da0ac5ed9f3b3, user_id=2781cd580c12435baee026b6c2e6c725, role_id=3255f77192524a16bc405173106871fc)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:32:20.957760 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68d141b-3df7-4899-9bf7-91207f164566 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:32:21.001575 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1333/2649] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:32:20 2022] PUT /identity/v3/projects/cc6164322c9f4826b96da0ac5ed9f3b3/users/2781cd580c12435baee026b6c2e6c725/roles/3255f77192524a16bc405173106871fc => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:32:21.014529 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e33b75-dea9-4a2d-af9e-fce98c3b1a86 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:21.015032 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e33b75-dea9-4a2d-af9e-fce98c3b1a86 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:21.015032 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e33b75-dea9-4a2d-af9e-fce98c3b1a86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:21.042677 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-73e33b75-dea9-4a2d-af9e-fce98c3b1a86 None None] MFA Rules not processed for user `2781cd580c12435baee026b6c2e6c725`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:32:21.044675 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-73e33b75-dea9-4a2d-af9e-fce98c3b1a86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:32:21.106411 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1317/2650] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:32:21 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:32:21.110931 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68d141b-3df7-4899-9bf7-91207f164566 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:32:21.141670 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:DELETE=12,keystone:SELECT=84,keystone:INSERT=4 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:32:21.178964 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68d141b-3df7-4899-9bf7-91207f164566 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:32:21.186008 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d58146b-4f3e-4e6f-8e64-6cbcba08daac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:32:21.187508 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d58146b-4f3e-4e6f-8e64-6cbcba08daac admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:21.187508 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d58146b-4f3e-4e6f-8e64-6cbcba08daac admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:21.195637 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d58146b-4f3e-4e6f-8e64-6cbcba08daac admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:21.195637 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d58146b-4f3e-4e6f-8e64-6cbcba08daac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:32:21.195637 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d58146b-4f3e-4e6f-8e64-6cbcba08daac admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:32:21.290546 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1334/2651] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:32:21 2022] POST /identity/v3/projects => generated 387 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:32:21.294635 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e33b75-dea9-4a2d-af9e-fce98c3b1a86 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:32:21.305509 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e33b75-dea9-4a2d-af9e-fce98c3b1a86 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:32:21.315396 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:32:21.366556 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24da944a-65e8-4546-933f-be841b3a1844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:32:21.367486 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24da944a-65e8-4546-933f-be841b3a1844 admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:21.367771 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24da944a-65e8-4546-933f-be841b3a1844 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:21.368048 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24da944a-65e8-4546-933f-be841b3a1844 admin admin] PATH_INFO: `/v3/users` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:21.368706 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24da944a-65e8-4546-933f-be841b3a1844 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:32:21.370673 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24da944a-65e8-4546-933f-be841b3a1844 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:32:21.457819 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1318/2652] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:32:21 2022] POST /identity/v3/users => generated 300 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:32:21.459195 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d58146b-4f3e-4e6f-8e64-6cbcba08daac admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:32:21.470595 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d58146b-4f3e-4e6f-8e64-6cbcba08daac admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:32:21.476506 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd816d4-9b40-4ee8-81a8-ec30c1cb4baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:32:21.476506 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd816d4-9b40-4ee8-81a8-ec30c1cb4baa admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:21.477034 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd816d4-9b40-4ee8-81a8-ec30c1cb4baa admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:21.477634 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd816d4-9b40-4ee8-81a8-ec30c1cb4baa admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:21.478144 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd816d4-9b40-4ee8-81a8-ec30c1cb4baa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:32:21.480316 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd816d4-9b40-4ee8-81a8-ec30c1cb4baa admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:32:21.494476 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1335/2653] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:32:21 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:32:21.497374 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24da944a-65e8-4546-933f-be841b3a1844 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:32:21.509141 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24da944a-65e8-4546-933f-be841b3a1844 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:32:21.514707 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f0d93d-0f9b-4d78-9d16-2e3bb72c3a88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:32:21.522992 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f0d93d-0f9b-4d78-9d16-2e3bb72c3a88 admin admin] REQUEST_METHOD: `PUT` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:21.522992 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f0d93d-0f9b-4d78-9d16-2e3bb72c3a88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:21.522992 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f0d93d-0f9b-4d78-9d16-2e3bb72c3a88 admin admin] PATH_INFO: `/v3/projects/3edbf0131c8d4406bf44c3343986a822/users/46c053eaa78445aa8c853d26df4f451b/roles/62f19ea272b447ccaab1576cd9bf4fd9` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:21.542978 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f0d93d-0f9b-4d78-9d16-2e3bb72c3a88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3edbf0131c8d4406bf44c3343986a822, user_id=46c053eaa78445aa8c853d26df4f451b, role_id=62f19ea272b447ccaab1576cd9bf4fd9)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:32:21.546075 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f0d93d-0f9b-4d78-9d16-2e3bb72c3a88 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:32:21.640895 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1319/2654] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:32:21 2022] PUT /identity/v3/projects/3edbf0131c8d4406bf44c3343986a822/users/46c053eaa78445aa8c853d26df4f451b/roles/62f19ea272b447ccaab1576cd9bf4fd9 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:32:21.644724 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd816d4-9b40-4ee8-81a8-ec30c1cb4baa admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:32:21.663171 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd816d4-9b40-4ee8-81a8-ec30c1cb4baa admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:32:21.727121 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949e1b43-c4b6-49c2-92eb-325cc38b2d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:32:21.727739 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949e1b43-c4b6-49c2-92eb-325cc38b2d7b admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:21.728023 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949e1b43-c4b6-49c2-92eb-325cc38b2d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:21.728299 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949e1b43-c4b6-49c2-92eb-325cc38b2d7b admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:21.728946 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949e1b43-c4b6-49c2-92eb-325cc38b2d7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:32:21.731080 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949e1b43-c4b6-49c2-92eb-325cc38b2d7b admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:32:21.741638 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1336/2655] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:32:21 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:32:21.746562 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f0d93d-0f9b-4d78-9d16-2e3bb72c3a88 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:32:21.757191 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f0d93d-0f9b-4d78-9d16-2e3bb72c3a88 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:32:21.762507 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1739032-343e-4a96-92ce-aa60d2aedaf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:32:21.763158 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1739032-343e-4a96-92ce-aa60d2aedaf9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:21.763438 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1739032-343e-4a96-92ce-aa60d2aedaf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:21.763708 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1739032-343e-4a96-92ce-aa60d2aedaf9 admin admin] PATH_INFO: `/v3/projects/3edbf0131c8d4406bf44c3343986a822/users/46c053eaa78445aa8c853d26df4f451b/roles/62f19ea272b447ccaab1576cd9bf4fd9` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:21.765682 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1739032-343e-4a96-92ce-aa60d2aedaf9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3edbf0131c8d4406bf44c3343986a822, user_id=46c053eaa78445aa8c853d26df4f451b, role_id=62f19ea272b447ccaab1576cd9bf4fd9)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:32:21.768351 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1739032-343e-4a96-92ce-aa60d2aedaf9 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:32:21.779212 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1320/2656] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:32:21 2022] PUT /identity/v3/projects/3edbf0131c8d4406bf44c3343986a822/users/46c053eaa78445aa8c853d26df4f451b/roles/62f19ea272b447ccaab1576cd9bf4fd9 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:32:21.790038 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df1329e-29e7-45a4-9f3a-440b9a331d5b None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:21.790384 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df1329e-29e7-45a4-9f3a-440b9a331d5b None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:21.790660 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df1329e-29e7-45a4-9f3a-440b9a331d5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:21.820876 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-6df1329e-29e7-45a4-9f3a-440b9a331d5b None None] MFA Rules not processed for user `46c053eaa78445aa8c853d26df4f451b`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:32:21.822161 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-6df1329e-29e7-45a4-9f3a-440b9a331d5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:32:21.891864 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1337/2657] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:32:21 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:32:21.903495 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1739032-343e-4a96-92ce-aa60d2aedaf9 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:32:21.905259 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-e1739032-343e-4a96-92ce-aa60d2aedaf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:32:21.965567 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1739032-343e-4a96-92ce-aa60d2aedaf9 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:32:21.969233 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9de8f5f-d84f-4f96-a682-80aa2c406622 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60193ba9e1c14df3bc9eebcc6f28f2d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:32:21.969839 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9de8f5f-d84f-4f96-a682-80aa2c406622 service cinder] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:21.970116 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9de8f5f-d84f-4f96-a682-80aa2c406622 service cinder] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:21.970428 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9de8f5f-d84f-4f96-a682-80aa2c406622 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:21.979560 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9de8f5f-d84f-4f96-a682-80aa2c406622 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:32:21.981512 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9de8f5f-d84f-4f96-a682-80aa2c406622 service cinder] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:32:21.991048 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1321/2658] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:32:21 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:32:23.068149 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df1329e-29e7-45a4-9f3a-440b9a331d5b None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:32:23.095820 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:32:23.134184 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df1329e-29e7-45a4-9f3a-440b9a331d5b None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:32:23.139846 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83926a3-81ad-442b-a6d1-545f561eaa98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:32:23.141024 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83926a3-81ad-442b-a6d1-545f561eaa98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:23.141307 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83926a3-81ad-442b-a6d1-545f561eaa98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:23.141588 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83926a3-81ad-442b-a6d1-545f561eaa98 admin admin] PATH_INFO: `/v3/users/2781cd580c12435baee026b6c2e6c725` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:23.142769 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83926a3-81ad-442b-a6d1-545f561eaa98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2781cd580c12435baee026b6c2e6c725)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:32:23.144701 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83926a3-81ad-442b-a6d1-545f561eaa98 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:32:23.291035 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-c83926a3-81ad-442b-a6d1-545f561eaa98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2781cd580c12435baee026b6c2e6c725'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:23.357328 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-c83926a3-81ad-442b-a6d1-545f561eaa98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2781cd580c12435baee026b6c2e6c725'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:23.357960 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-c83926a3-81ad-442b-a6d1-545f561eaa98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2781cd580c12435baee026b6c2e6c725'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:23.365655 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-c83926a3-81ad-442b-a6d1-545f561eaa98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2781cd580c12435baee026b6c2e6c725'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:23.378913 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-c83926a3-81ad-442b-a6d1-545f561eaa98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2781cd580c12435baee026b6c2e6c725'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:23.385055 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1338/2659] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:32:23 2022] DELETE /identity/v3/users/2781cd580c12435baee026b6c2e6c725 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:32:23.389787 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27953e4-9fe8-44de-844a-8f9d0084f45e None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:23.390262 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27953e4-9fe8-44de-844a-8f9d0084f45e None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:23.390406 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27953e4-9fe8-44de-844a-8f9d0084f45e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:23.409163 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-b27953e4-9fe8-44de-844a-8f9d0084f45e None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:32:23.410408 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-b27953e4-9fe8-44de-844a-8f9d0084f45e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:32:23.470292 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1322/2660] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:32:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:32:23.480936 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83926a3-81ad-442b-a6d1-545f561eaa98 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:32:23.482398 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-c83926a3-81ad-442b-a6d1-545f561eaa98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:32:23.541455 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83926a3-81ad-442b-a6d1-545f561eaa98 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:32:23.545695 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f0c7b6-0b7b-460e-9465-f0bbe5d6160c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:32:23.546343 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f0c7b6-0b7b-460e-9465-f0bbe5d6160c service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:23.546628 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f0c7b6-0b7b-460e-9465-f0bbe5d6160c service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:23.546900 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f0c7b6-0b7b-460e-9465-f0bbe5d6160c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:23.556322 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f0c7b6-0b7b-460e-9465-f0bbe5d6160c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:32:23.557752 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f0c7b6-0b7b-460e-9465-f0bbe5d6160c service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:32:23.569012 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1339/2661] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:32:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:32:23.838662 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27953e4-9fe8-44de-844a-8f9d0084f45e None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:32:23.840619 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-b27953e4-9fe8-44de-844a-8f9d0084f45e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:32:23.851632 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:32:23.918991 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27953e4-9fe8-44de-844a-8f9d0084f45e None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:32:23.924575 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb32a40-e2d8-47b6-bfa9-7013ff6ec1af service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cd171bf4df94f8fab397e42835feeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:32:23.925303 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb32a40-e2d8-47b6-bfa9-7013ff6ec1af service nova] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:23.925590 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb32a40-e2d8-47b6-bfa9-7013ff6ec1af service nova] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:23.925864 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb32a40-e2d8-47b6-bfa9-7013ff6ec1af service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:23.927724 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-bcb32a40-e2d8-47b6-bfa9-7013ff6ec1af service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:32:23.988973 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb32a40-e2d8-47b6-bfa9-7013ff6ec1af service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:32:23.990656 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb32a40-e2d8-47b6-bfa9-7013ff6ec1af service nova] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:32:24.000302 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1323/2662] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:32:23 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:32:24.352609 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f0c7b6-0b7b-460e-9465-f0bbe5d6160c service neutron] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:32:24.354098 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-85f0c7b6-0b7b-460e-9465-f0bbe5d6160c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:32:24.367759 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f0c7b6-0b7b-460e-9465-f0bbe5d6160c service neutron] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:32:24.372980 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fabe17e-38d1-4bd1-b42b-d54e50b9c88d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:32:24.374047 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fabe17e-38d1-4bd1-b42b-d54e50b9c88d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:24.374047 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fabe17e-38d1-4bd1-b42b-d54e50b9c88d admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:24.374250 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fabe17e-38d1-4bd1-b42b-d54e50b9c88d admin admin] PATH_INFO: `/v3/projects/cc6164322c9f4826b96da0ac5ed9f3b3` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:24.375631 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fabe17e-38d1-4bd1-b42b-d54e50b9c88d admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc6164322c9f4826b96da0ac5ed9f3b3)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:32:24.377399 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fabe17e-38d1-4bd1-b42b-d54e50b9c88d admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:32:24.463887 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:32:24.470736 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-2fabe17e-38d1-4bd1-b42b-d54e50b9c88d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc6164322c9f4826b96da0ac5ed9f3b3', 'request_id': 'req-2fabe17e-38d1-4bd1-b42b-d54e50b9c88d'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:24.521941 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-2fabe17e-38d1-4bd1-b42b-d54e50b9c88d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc6164322c9f4826b96da0ac5ed9f3b3', 'request_id': 'req-2fabe17e-38d1-4bd1-b42b-d54e50b9c88d'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:24.527818 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-2fabe17e-38d1-4bd1-b42b-d54e50b9c88d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc6164322c9f4826b96da0ac5ed9f3b3', 'request_id': 'req-2fabe17e-38d1-4bd1-b42b-d54e50b9c88d'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:24.539504 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-2fabe17e-38d1-4bd1-b42b-d54e50b9c88d admin admin] The token cache is being invalidate because project cc6164322c9f4826b96da0ac5ed9f3b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:32:24.539796 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-2fabe17e-38d1-4bd1-b42b-d54e50b9c88d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:24.540453 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-2fabe17e-38d1-4bd1-b42b-d54e50b9c88d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:24.542075 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1340/2663] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:32:24 2022] DELETE /identity/v3/projects/cc6164322c9f4826b96da0ac5ed9f3b3 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:32:24.546121 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb32a40-e2d8-47b6-bfa9-7013ff6ec1af service nova] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:32:24.556466 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb32a40-e2d8-47b6-bfa9-7013ff6ec1af service nova] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:32:24.612986 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade146ff-742c-4c71-962d-0587eeac1735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:32:24.613622 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade146ff-742c-4c71-962d-0587eeac1735 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:24.613908 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade146ff-742c-4c71-962d-0587eeac1735 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:24.614181 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade146ff-742c-4c71-962d-0587eeac1735 admin admin] PATH_INFO: `/v3/users/46c053eaa78445aa8c853d26df4f451b` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:24.615336 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade146ff-742c-4c71-962d-0587eeac1735 admin admin] RBAC: Authorizing `identity:delete_user(user_id=46c053eaa78445aa8c853d26df4f451b)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:32:24.617253 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade146ff-742c-4c71-962d-0587eeac1735 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:32:24.780696 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-ade146ff-742c-4c71-962d-0587eeac1735 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46c053eaa78445aa8c853d26df4f451b'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:24.782342 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:32:24.847536 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-ade146ff-742c-4c71-962d-0587eeac1735 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46c053eaa78445aa8c853d26df4f451b'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:24.848159 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-ade146ff-742c-4c71-962d-0587eeac1735 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46c053eaa78445aa8c853d26df4f451b'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:24.848771 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-ade146ff-742c-4c71-962d-0587eeac1735 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46c053eaa78445aa8c853d26df4f451b'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:24.861760 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-ade146ff-742c-4c71-962d-0587eeac1735 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46c053eaa78445aa8c853d26df4f451b'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:24.874568 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1324/2664] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:32:24 2022] DELETE /identity/v3/users/46c053eaa78445aa8c853d26df4f451b => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:32:25.238830 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fabe17e-38d1-4bd1-b42b-d54e50b9c88d admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:32:25.240657 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-2fabe17e-38d1-4bd1-b42b-d54e50b9c88d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:32:25.255860 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fabe17e-38d1-4bd1-b42b-d54e50b9c88d admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:32:25.314986 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f9febf-3341-4863-85d8-d7fcec73ed6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:32:25.315623 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f9febf-3341-4863-85d8-d7fcec73ed6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:25.315898 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f9febf-3341-4863-85d8-d7fcec73ed6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:25.316173 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f9febf-3341-4863-85d8-d7fcec73ed6e admin admin] PATH_INFO: `/v3/projects/3edbf0131c8d4406bf44c3343986a822` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:25.317283 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f9febf-3341-4863-85d8-d7fcec73ed6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3edbf0131c8d4406bf44c3343986a822)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:32:25.319207 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f9febf-3341-4863-85d8-d7fcec73ed6e admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:32:25.396395 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-72f9febf-3341-4863-85d8-d7fcec73ed6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3edbf0131c8d4406bf44c3343986a822', 'request_id': 'req-72f9febf-3341-4863-85d8-d7fcec73ed6e'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:25.436385 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-72f9febf-3341-4863-85d8-d7fcec73ed6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3edbf0131c8d4406bf44c3343986a822', 'request_id': 'req-72f9febf-3341-4863-85d8-d7fcec73ed6e'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:25.442060 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-72f9febf-3341-4863-85d8-d7fcec73ed6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3edbf0131c8d4406bf44c3343986a822', 'request_id': 'req-72f9febf-3341-4863-85d8-d7fcec73ed6e'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:25.453284 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-72f9febf-3341-4863-85d8-d7fcec73ed6e admin admin] The token cache is being invalidate because project 3edbf0131c8d4406bf44c3343986a822 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:32:25.453573 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-72f9febf-3341-4863-85d8-d7fcec73ed6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:25.454201 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-72f9febf-3341-4863-85d8-d7fcec73ed6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:25.455744 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1341/2665] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:32:25 2022] DELETE /identity/v3/projects/3edbf0131c8d4406bf44c3343986a822 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:32:25.460607 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aa7f02-950a-4c8d-b8f1-aa36c742b172 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:25.460899 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aa7f02-950a-4c8d-b8f1-aa36c742b172 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:25.461199 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aa7f02-950a-4c8d-b8f1-aa36c742b172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:25.481371 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-74aa7f02-950a-4c8d-b8f1-aa36c742b172 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:32:25.482618 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-74aa7f02-950a-4c8d-b8f1-aa36c742b172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:32:25.540444 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1325/2666] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:32:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:32:25.552443 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d25c086-e963-456d-aaa6-a93d6f7662ee None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:25.552443 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d25c086-e963-456d-aaa6-a93d6f7662ee None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:25.552443 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d25c086-e963-456d-aaa6-a93d6f7662ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:25.569033 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-8d25c086-e963-456d-aaa6-a93d6f7662ee None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:32:25.570258 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-8d25c086-e963-456d-aaa6-a93d6f7662ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:32:25.579032 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1342/2667] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:32:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:32:25.583410 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74aa7f02-950a-4c8d-b8f1-aa36c742b172 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:32:25.597417 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74aa7f02-950a-4c8d-b8f1-aa36c742b172 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:32:25.602790 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff98d29-a3c1-4152-9055-e9a4df263acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:32:25.603396 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff98d29-a3c1-4152-9055-e9a4df263acf admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:25.603678 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff98d29-a3c1-4152-9055-e9a4df263acf admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:25.603947 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff98d29-a3c1-4152-9055-e9a4df263acf admin admin] PATH_INFO: `/v3/domains` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:25.604594 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff98d29-a3c1-4152-9055-e9a4df263acf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:32:25.604899 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff98d29-a3c1-4152-9055-e9a4df263acf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:32:25.606602 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff98d29-a3c1-4152-9055-e9a4df263acf admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:32:25.618334 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1326/2668] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Aug 25 03:32:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:32:25.622786 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d25c086-e963-456d-aaa6-a93d6f7662ee None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:32:25.632682 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d25c086-e963-456d-aaa6-a93d6f7662ee None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:32:25.638650 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634967f9-536a-4c85-b178-57f74a5e00ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:32:25.639260 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634967f9-536a-4c85-b178-57f74a5e00ef admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:25.639539 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634967f9-536a-4c85-b178-57f74a5e00ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:25.639809 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634967f9-536a-4c85-b178-57f74a5e00ef admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:25.640431 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634967f9-536a-4c85-b178-57f74a5e00ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:32:25.642346 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634967f9-536a-4c85-b178-57f74a5e00ef admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:32:25.741749 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1343/2669] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:32:25 2022] POST /identity/v3/projects => generated 377 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:32:25.746134 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff98d29-a3c1-4152-9055-e9a4df263acf admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:32:25.756306 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff98d29-a3c1-4152-9055-e9a4df263acf admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:32:25.813682 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af8d721-0f92-4db6-b37f-1de30cc99310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:32:25.814229 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af8d721-0f92-4db6-b37f-1de30cc99310 admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:25.814541 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af8d721-0f92-4db6-b37f-1de30cc99310 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:25.814819 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af8d721-0f92-4db6-b37f-1de30cc99310 admin admin] PATH_INFO: `/v3/users` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:25.815449 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af8d721-0f92-4db6-b37f-1de30cc99310 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:32:25.817361 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af8d721-0f92-4db6-b37f-1de30cc99310 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:32:25.893820 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1327/2670] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:32:25 2022] POST /identity/v3/users => generated 289 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:32:25.898219 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634967f9-536a-4c85-b178-57f74a5e00ef admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:32:25.908398 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634967f9-536a-4c85-b178-57f74a5e00ef admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:32:25.912999 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495ec0ae-6cf4-4c67-988d-06294217b048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:32:25.913594 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495ec0ae-6cf4-4c67-988d-06294217b048 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:25.913875 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495ec0ae-6cf4-4c67-988d-06294217b048 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:25.914151 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495ec0ae-6cf4-4c67-988d-06294217b048 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:25.914836 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495ec0ae-6cf4-4c67-988d-06294217b048 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:32:25.916436 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495ec0ae-6cf4-4c67-988d-06294217b048 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:32:25.926403 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1344/2671] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:32:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:32:25.930232 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af8d721-0f92-4db6-b37f-1de30cc99310 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:32:25.941542 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af8d721-0f92-4db6-b37f-1de30cc99310 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:32:25.946170 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f63f64-9261-4585-805a-1070a8ad7e11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:32:25.946802 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f63f64-9261-4585-805a-1070a8ad7e11 admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:25.947086 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f63f64-9261-4585-805a-1070a8ad7e11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:25.947364 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f63f64-9261-4585-805a-1070a8ad7e11 admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:25.948009 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f63f64-9261-4585-805a-1070a8ad7e11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:32:25.950242 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f63f64-9261-4585-805a-1070a8ad7e11 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:32:25.960604 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1328/2672] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:32:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:32:25.964747 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495ec0ae-6cf4-4c67-988d-06294217b048 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:32:25.974469 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495ec0ae-6cf4-4c67-988d-06294217b048 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:32:25.979048 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaa5c3f-4685-4ab1-b127-5b51f0c60e3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:32:25.979687 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaa5c3f-4685-4ab1-b127-5b51f0c60e3f admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:25.979970 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaa5c3f-4685-4ab1-b127-5b51f0c60e3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:25.980272 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaa5c3f-4685-4ab1-b127-5b51f0c60e3f admin admin] PATH_INFO: `/v3/projects/e8cba91b50da4e0e815c820b142140f1/users/c02b80fc3e5f45f884b691e575c18dc3/roles/3255f77192524a16bc405173106871fc` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:25.995570 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaa5c3f-4685-4ab1-b127-5b51f0c60e3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8cba91b50da4e0e815c820b142140f1, user_id=c02b80fc3e5f45f884b691e575c18dc3, role_id=3255f77192524a16bc405173106871fc)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:32:25.998919 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaa5c3f-4685-4ab1-b127-5b51f0c60e3f admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:32:26.001156 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:32:26.065367 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1345/2673] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:32:25 2022] PUT /identity/v3/projects/e8cba91b50da4e0e815c820b142140f1/users/c02b80fc3e5f45f884b691e575c18dc3/roles/3255f77192524a16bc405173106871fc => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:32:26.077276 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cb7353-b4c9-4d3a-a680-b5de19799608 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:26.077565 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cb7353-b4c9-4d3a-a680-b5de19799608 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:26.077843 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cb7353-b4c9-4d3a-a680-b5de19799608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:26.104641 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-35cb7353-b4c9-4d3a-a680-b5de19799608 None None] MFA Rules not processed for user `c02b80fc3e5f45f884b691e575c18dc3`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:32:26.105987 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-35cb7353-b4c9-4d3a-a680-b5de19799608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:32:26.124130 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:32:26.170019 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1329/2674] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:32:26 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:32:26.181085 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaa5c3f-4685-4ab1-b127-5b51f0c60e3f admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:32:26.182825 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-5eaa5c3f-4685-4ab1-b127-5b51f0c60e3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:32:26.262210 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaa5c3f-4685-4ab1-b127-5b51f0c60e3f admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:32:26.266058 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124ff5ca-0be4-4e2e-8717-6eecca8c0556 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f89089b390c4b6599e54000c0fa082a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:32:26.266699 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124ff5ca-0be4-4e2e-8717-6eecca8c0556 service glance] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:26.266981 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124ff5ca-0be4-4e2e-8717-6eecca8c0556 service glance] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:26.267257 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124ff5ca-0be4-4e2e-8717-6eecca8c0556 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:26.275866 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124ff5ca-0be4-4e2e-8717-6eecca8c0556 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:32:26.277359 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124ff5ca-0be4-4e2e-8717-6eecca8c0556 service glance] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:32:26.286052 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1346/2675] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:32:26 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:32:36.253734 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=18 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:32:36.282083 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:INSERT=1,keystone:SELECT=37 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:32:48.749256 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cb7353-b4c9-4d3a-a680-b5de19799608 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:32:48.810641 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cb7353-b4c9-4d3a-a680-b5de19799608 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:32:48.815749 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312633f2-04c1-47f7-b5f4-372b719bbe0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:32:48.816374 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312633f2-04c1-47f7-b5f4-372b719bbe0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:48.816655 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312633f2-04c1-47f7-b5f4-372b719bbe0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:48.816931 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312633f2-04c1-47f7-b5f4-372b719bbe0a admin admin] PATH_INFO: `/v3/users/c02b80fc3e5f45f884b691e575c18dc3` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:48.818038 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312633f2-04c1-47f7-b5f4-372b719bbe0a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c02b80fc3e5f45f884b691e575c18dc3)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:32:48.819987 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312633f2-04c1-47f7-b5f4-372b719bbe0a admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:32:49.152566 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-312633f2-04c1-47f7-b5f4-372b719bbe0a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c02b80fc3e5f45f884b691e575c18dc3'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:49.253161 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-312633f2-04c1-47f7-b5f4-372b719bbe0a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c02b80fc3e5f45f884b691e575c18dc3'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:49.253803 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-312633f2-04c1-47f7-b5f4-372b719bbe0a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c02b80fc3e5f45f884b691e575c18dc3'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:49.254445 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-312633f2-04c1-47f7-b5f4-372b719bbe0a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c02b80fc3e5f45f884b691e575c18dc3'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:49.267041 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-312633f2-04c1-47f7-b5f4-372b719bbe0a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c02b80fc3e5f45f884b691e575c18dc3'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:49.279086 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1330/2676] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:32:48 2022] DELETE /identity/v3/users/c02b80fc3e5f45f884b691e575c18dc3 => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:32:49.284124 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af491d19-7d46-481f-8cf0-8b13938cd99a None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:49.284632 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af491d19-7d46-481f-8cf0-8b13938cd99a None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:49.284745 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af491d19-7d46-481f-8cf0-8b13938cd99a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:49.304175 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-af491d19-7d46-481f-8cf0-8b13938cd99a None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:32:49.305375 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-af491d19-7d46-481f-8cf0-8b13938cd99a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:32:49.363191 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1347/2677] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:32:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:32:49.372893 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312633f2-04c1-47f7-b5f4-372b719bbe0a admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:32:49.374356 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-312633f2-04c1-47f7-b5f4-372b719bbe0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:32:49.435028 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312633f2-04c1-47f7-b5f4-372b719bbe0a admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:32:49.438559 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0812cf69-ae32-456a-a572-556d0efbd87d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:32:49.439173 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0812cf69-ae32-456a-a572-556d0efbd87d service neutron] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:49.439455 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0812cf69-ae32-456a-a572-556d0efbd87d service neutron] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:49.439731 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0812cf69-ae32-456a-a572-556d0efbd87d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:49.448475 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0812cf69-ae32-456a-a572-556d0efbd87d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:32:49.449898 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0812cf69-ae32-456a-a572-556d0efbd87d service neutron] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:32:49.459889 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1331/2678] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:32:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:32:49.895261 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af491d19-7d46-481f-8cf0-8b13938cd99a None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:32:49.897093 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-af491d19-7d46-481f-8cf0-8b13938cd99a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:32:49.911073 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af491d19-7d46-481f-8cf0-8b13938cd99a None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:32:49.916058 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4f1e78-1af7-4d7b-b860-49219a0ff6ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:32:49.916697 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4f1e78-1af7-4d7b-b860-49219a0ff6ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:49.916996 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4f1e78-1af7-4d7b-b860-49219a0ff6ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:49.917238 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4f1e78-1af7-4d7b-b860-49219a0ff6ac admin admin] PATH_INFO: `/v3/projects/e8cba91b50da4e0e815c820b142140f1` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:49.919730 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4f1e78-1af7-4d7b-b860-49219a0ff6ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8cba91b50da4e0e815c820b142140f1)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:32:49.925597 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4f1e78-1af7-4d7b-b860-49219a0ff6ac admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:32:50.047049 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-ba4f1e78-1af7-4d7b-b860-49219a0ff6ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8cba91b50da4e0e815c820b142140f1', 'request_id': 'req-ba4f1e78-1af7-4d7b-b860-49219a0ff6ac'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:50.116133 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-ba4f1e78-1af7-4d7b-b860-49219a0ff6ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8cba91b50da4e0e815c820b142140f1', 'request_id': 'req-ba4f1e78-1af7-4d7b-b860-49219a0ff6ac'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:50.122420 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-ba4f1e78-1af7-4d7b-b860-49219a0ff6ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8cba91b50da4e0e815c820b142140f1', 'request_id': 'req-ba4f1e78-1af7-4d7b-b860-49219a0ff6ac'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:50.134246 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-ba4f1e78-1af7-4d7b-b860-49219a0ff6ac admin admin] The token cache is being invalidate because project e8cba91b50da4e0e815c820b142140f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:32:50.134564 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-ba4f1e78-1af7-4d7b-b860-49219a0ff6ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:50.135206 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-ba4f1e78-1af7-4d7b-b860-49219a0ff6ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:50.136692 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1348/2679] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:32:49 2022] DELETE /identity/v3/projects/e8cba91b50da4e0e815c820b142140f1 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:32:50.142303 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e060f5-2756-4959-a4cf-4f06d90c6efb None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:50.142806 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e060f5-2756-4959-a4cf-4f06d90c6efb None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:50.142925 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e060f5-2756-4959-a4cf-4f06d90c6efb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:50.161949 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-a5e060f5-2756-4959-a4cf-4f06d90c6efb None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:32:50.163181 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-a5e060f5-2756-4959-a4cf-4f06d90c6efb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:32:50.180497 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:32:50.221884 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1332/2680] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:32:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:32:50.229665 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edabac2-0a67-4078-a23a-2b03e9235cd7 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:50.233148 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edabac2-0a67-4078-a23a-2b03e9235cd7 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:50.233148 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edabac2-0a67-4078-a23a-2b03e9235cd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:50.250714 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-2edabac2-0a67-4078-a23a-2b03e9235cd7 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:32:50.251915 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-2edabac2-0a67-4078-a23a-2b03e9235cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:32:50.259794 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1349/2681] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:32:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:32:50.263880 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e060f5-2756-4959-a4cf-4f06d90c6efb None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:32:50.273528 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e060f5-2756-4959-a4cf-4f06d90c6efb None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:32:50.278156 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df4f0d4-aa1a-464a-9743-47667b70ea45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:32:50.278791 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df4f0d4-aa1a-464a-9743-47667b70ea45 admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:50.279071 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df4f0d4-aa1a-464a-9743-47667b70ea45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:50.279349 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df4f0d4-aa1a-464a-9743-47667b70ea45 admin admin] PATH_INFO: `/v3/domains` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:50.280022 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df4f0d4-aa1a-464a-9743-47667b70ea45 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:32:50.280327 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df4f0d4-aa1a-464a-9743-47667b70ea45 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:32:50.281928 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df4f0d4-aa1a-464a-9743-47667b70ea45 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:32:50.293408 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1333/2682] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Aug 25 03:32:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:32:50.297888 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edabac2-0a67-4078-a23a-2b03e9235cd7 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:32:50.306994 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edabac2-0a67-4078-a23a-2b03e9235cd7 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:32:50.312520 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5c5676-b3d1-4640-85be-13db2d362a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:32:50.313119 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5c5676-b3d1-4640-85be-13db2d362a81 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:50.313400 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5c5676-b3d1-4640-85be-13db2d362a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:50.313673 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5c5676-b3d1-4640-85be-13db2d362a81 admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:50.314336 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5c5676-b3d1-4640-85be-13db2d362a81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:32:50.316218 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5c5676-b3d1-4640-85be-13db2d362a81 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:32:50.381130 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1350/2683] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:32:50 2022] POST /identity/v3/projects => generated 383 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:32:50.385629 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df4f0d4-aa1a-464a-9743-47667b70ea45 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:32:50.395601 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df4f0d4-aa1a-464a-9743-47667b70ea45 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:32:50.451944 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4efb58-56f1-4ac3-a5d6-cf11e9f7ba68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:32:50.452549 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4efb58-56f1-4ac3-a5d6-cf11e9f7ba68 admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:50.452830 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4efb58-56f1-4ac3-a5d6-cf11e9f7ba68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:50.453107 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4efb58-56f1-4ac3-a5d6-cf11e9f7ba68 admin admin] PATH_INFO: `/v3/users` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:50.453737 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4efb58-56f1-4ac3-a5d6-cf11e9f7ba68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:32:50.455671 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4efb58-56f1-4ac3-a5d6-cf11e9f7ba68 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:32:50.527255 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1334/2684] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:32:50 2022] POST /identity/v3/users => generated 292 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:32:50.531631 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5c5676-b3d1-4640-85be-13db2d362a81 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:32:50.542790 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5c5676-b3d1-4640-85be-13db2d362a81 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:32:50.548040 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d480a3ae-10e5-4572-b5e1-5d3ce376b1ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:32:50.548573 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d480a3ae-10e5-4572-b5e1-5d3ce376b1ac admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:50.548849 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d480a3ae-10e5-4572-b5e1-5d3ce376b1ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:50.549150 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d480a3ae-10e5-4572-b5e1-5d3ce376b1ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:50.549781 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d480a3ae-10e5-4572-b5e1-5d3ce376b1ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:32:50.551404 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d480a3ae-10e5-4572-b5e1-5d3ce376b1ac admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:32:50.562403 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1351/2685] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:32:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:32:50.566257 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4efb58-56f1-4ac3-a5d6-cf11e9f7ba68 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:32:50.576223 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4efb58-56f1-4ac3-a5d6-cf11e9f7ba68 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:32:50.581066 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d983b413-92e3-482c-b500-fac81fac552d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:32:50.582838 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d983b413-92e3-482c-b500-fac81fac552d admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:50.582838 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d983b413-92e3-482c-b500-fac81fac552d admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:50.582838 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d983b413-92e3-482c-b500-fac81fac552d admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:50.583117 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d983b413-92e3-482c-b500-fac81fac552d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:32:50.584663 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d983b413-92e3-482c-b500-fac81fac552d admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:32:50.595360 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1335/2686] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:32:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:32:50.599400 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d480a3ae-10e5-4572-b5e1-5d3ce376b1ac admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:32:50.609850 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d480a3ae-10e5-4572-b5e1-5d3ce376b1ac admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:32:50.614499 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cc58c9-7674-45e6-95d3-e40340c43fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:32:50.617999 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cc58c9-7674-45e6-95d3-e40340c43fd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:50.617999 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cc58c9-7674-45e6-95d3-e40340c43fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:50.617999 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cc58c9-7674-45e6-95d3-e40340c43fd0 admin admin] PATH_INFO: `/v3/projects/e4242585930e4257bfa57eba857ba4ae/users/800d3b24cafe496f99e4834c59d854dc/roles/3255f77192524a16bc405173106871fc` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:50.629739 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cc58c9-7674-45e6-95d3-e40340c43fd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4242585930e4257bfa57eba857ba4ae, user_id=800d3b24cafe496f99e4834c59d854dc, role_id=3255f77192524a16bc405173106871fc)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:32:50.632390 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cc58c9-7674-45e6-95d3-e40340c43fd0 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:32:50.716063 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1352/2687] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:32:50 2022] PUT /identity/v3/projects/e4242585930e4257bfa57eba857ba4ae/users/800d3b24cafe496f99e4834c59d854dc/roles/3255f77192524a16bc405173106871fc => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:32:50.728190 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1b09ec-b5a4-44c3-a282-46852428a101 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:50.728482 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1b09ec-b5a4-44c3-a282-46852428a101 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:50.728760 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1b09ec-b5a4-44c3-a282-46852428a101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:50.756892 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-fa1b09ec-b5a4-44c3-a282-46852428a101 None None] MFA Rules not processed for user `800d3b24cafe496f99e4834c59d854dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:32:50.758089 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-fa1b09ec-b5a4-44c3-a282-46852428a101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:32:50.822272 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1336/2688] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:32:50 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:32:50.833336 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cc58c9-7674-45e6-95d3-e40340c43fd0 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:32:50.835048 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-72cc58c9-7674-45e6-95d3-e40340c43fd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:32:50.856779 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:32:50.891564 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cc58c9-7674-45e6-95d3-e40340c43fd0 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:32:50.895259 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bbbc73-3b67-40d3-8dd0-b63e9455f5a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60193ba9e1c14df3bc9eebcc6f28f2d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:32:50.895859 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bbbc73-3b67-40d3-8dd0-b63e9455f5a4 service cinder] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:50.896144 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bbbc73-3b67-40d3-8dd0-b63e9455f5a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:50.896418 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bbbc73-3b67-40d3-8dd0-b63e9455f5a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:50.906181 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bbbc73-3b67-40d3-8dd0-b63e9455f5a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:32:50.907724 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bbbc73-3b67-40d3-8dd0-b63e9455f5a4 service cinder] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:32:50.916521 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1353/2689] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:32:50 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:32:53.905369 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1b09ec-b5a4-44c3-a282-46852428a101 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:32:53.924650 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:32:53.968469 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1b09ec-b5a4-44c3-a282-46852428a101 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:32:53.972964 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbffb55a-8024-4c9a-bea5-2518e7c9d2f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:32:53.973601 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbffb55a-8024-4c9a-bea5-2518e7c9d2f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:53.973883 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbffb55a-8024-4c9a-bea5-2518e7c9d2f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:53.974160 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbffb55a-8024-4c9a-bea5-2518e7c9d2f0 admin admin] PATH_INFO: `/v3/users/800d3b24cafe496f99e4834c59d854dc` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:53.975309 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbffb55a-8024-4c9a-bea5-2518e7c9d2f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=800d3b24cafe496f99e4834c59d854dc)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:32:53.977262 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbffb55a-8024-4c9a-bea5-2518e7c9d2f0 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:32:54.141234 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-fbffb55a-8024-4c9a-bea5-2518e7c9d2f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '800d3b24cafe496f99e4834c59d854dc'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:54.218962 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-fbffb55a-8024-4c9a-bea5-2518e7c9d2f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '800d3b24cafe496f99e4834c59d854dc'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:54.219579 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-fbffb55a-8024-4c9a-bea5-2518e7c9d2f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '800d3b24cafe496f99e4834c59d854dc'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:54.220193 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-fbffb55a-8024-4c9a-bea5-2518e7c9d2f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '800d3b24cafe496f99e4834c59d854dc'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:54.232390 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-fbffb55a-8024-4c9a-bea5-2518e7c9d2f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '800d3b24cafe496f99e4834c59d854dc'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:54.244451 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1337/2690] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:32:53 2022] DELETE /identity/v3/users/800d3b24cafe496f99e4834c59d854dc => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:32:54.248970 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cee0a1-a4b4-44be-88e4-0100d25088c7 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:54.249458 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cee0a1-a4b4-44be-88e4-0100d25088c7 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:54.249560 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cee0a1-a4b4-44be-88e4-0100d25088c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:54.268463 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-c4cee0a1-a4b4-44be-88e4-0100d25088c7 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:32:54.269690 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-c4cee0a1-a4b4-44be-88e4-0100d25088c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:32:54.326834 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1354/2691] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:32:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:32:54.337023 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbffb55a-8024-4c9a-bea5-2518e7c9d2f0 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:32:54.338607 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-fbffb55a-8024-4c9a-bea5-2518e7c9d2f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:32:54.394178 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbffb55a-8024-4c9a-bea5-2518e7c9d2f0 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:32:54.397701 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679fd71c-3b21-46fd-9620-28dc09c5fe30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:32:54.398315 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679fd71c-3b21-46fd-9620-28dc09c5fe30 service neutron] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:54.398600 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679fd71c-3b21-46fd-9620-28dc09c5fe30 service neutron] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:54.398869 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679fd71c-3b21-46fd-9620-28dc09c5fe30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:54.408116 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679fd71c-3b21-46fd-9620-28dc09c5fe30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:32:54.409584 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679fd71c-3b21-46fd-9620-28dc09c5fe30 service neutron] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:32:54.419719 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1338/2692] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:32:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:32:54.841063 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cee0a1-a4b4-44be-88e4-0100d25088c7 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:32:54.841063 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-c4cee0a1-a4b4-44be-88e4-0100d25088c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:32:54.851220 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cee0a1-a4b4-44be-88e4-0100d25088c7 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:32:54.855874 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816de963-bd87-4095-99be-bd61ec4c95e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:32:54.856566 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816de963-bd87-4095-99be-bd61ec4c95e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:54.856843 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816de963-bd87-4095-99be-bd61ec4c95e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:54.857115 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816de963-bd87-4095-99be-bd61ec4c95e4 admin admin] PATH_INFO: `/v3/projects/e4242585930e4257bfa57eba857ba4ae` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:54.858867 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816de963-bd87-4095-99be-bd61ec4c95e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4242585930e4257bfa57eba857ba4ae)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:32:54.860867 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816de963-bd87-4095-99be-bd61ec4c95e4 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:32:54.945475 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-816de963-bd87-4095-99be-bd61ec4c95e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4242585930e4257bfa57eba857ba4ae', 'request_id': 'req-816de963-bd87-4095-99be-bd61ec4c95e4'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:55.019160 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-816de963-bd87-4095-99be-bd61ec4c95e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4242585930e4257bfa57eba857ba4ae', 'request_id': 'req-816de963-bd87-4095-99be-bd61ec4c95e4'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:55.025901 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-816de963-bd87-4095-99be-bd61ec4c95e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4242585930e4257bfa57eba857ba4ae', 'request_id': 'req-816de963-bd87-4095-99be-bd61ec4c95e4'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:55.038120 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-816de963-bd87-4095-99be-bd61ec4c95e4 admin admin] The token cache is being invalidate because project e4242585930e4257bfa57eba857ba4ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:32:55.038436 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-816de963-bd87-4095-99be-bd61ec4c95e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:55.039101 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-816de963-bd87-4095-99be-bd61ec4c95e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:32:55.040691 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1355/2693] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:32:54 2022] DELETE /identity/v3/projects/e4242585930e4257bfa57eba857ba4ae => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:32:55.045762 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac1e27b-892c-4fcf-8ff0-e027ba8cfba0 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:55.046952 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac1e27b-892c-4fcf-8ff0-e027ba8cfba0 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:55.046952 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac1e27b-892c-4fcf-8ff0-e027ba8cfba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:55.067777 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-dac1e27b-892c-4fcf-8ff0-e027ba8cfba0 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:32:55.068977 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-dac1e27b-892c-4fcf-8ff0-e027ba8cfba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:32:55.101599 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:32:55.126849 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1339/2694] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:32:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:32:55.189313 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2b5908-675d-4b7d-a957-cd900818f3b5 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:55.189703 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2b5908-675d-4b7d-a957-cd900818f3b5 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:55.190071 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2b5908-675d-4b7d-a957-cd900818f3b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:55.210866 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-dd2b5908-675d-4b7d-a957-cd900818f3b5 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:32:55.212018 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-dd2b5908-675d-4b7d-a957-cd900818f3b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:32:55.220360 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1356/2695] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:32:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:32:55.224620 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac1e27b-892c-4fcf-8ff0-e027ba8cfba0 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:32:55.234855 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac1e27b-892c-4fcf-8ff0-e027ba8cfba0 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:32:55.239328 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c576a2f-32b1-4760-b7bc-94b2464a1f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:32:55.239925 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c576a2f-32b1-4760-b7bc-94b2464a1f16 admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:55.240206 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c576a2f-32b1-4760-b7bc-94b2464a1f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:55.240480 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c576a2f-32b1-4760-b7bc-94b2464a1f16 admin admin] PATH_INFO: `/v3/domains` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:55.241124 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c576a2f-32b1-4760-b7bc-94b2464a1f16 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:32:55.241427 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c576a2f-32b1-4760-b7bc-94b2464a1f16 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:32:55.243071 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c576a2f-32b1-4760-b7bc-94b2464a1f16 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:32:55.254321 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1340/2696] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Aug 25 03:32:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:32:55.258347 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2b5908-675d-4b7d-a957-cd900818f3b5 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:32:55.267416 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2b5908-675d-4b7d-a957-cd900818f3b5 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:32:55.272234 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa847cdf-6cc3-45ec-a474-d4fcda263595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:32:55.272829 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa847cdf-6cc3-45ec-a474-d4fcda263595 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:55.273141 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa847cdf-6cc3-45ec-a474-d4fcda263595 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:55.273413 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa847cdf-6cc3-45ec-a474-d4fcda263595 admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:55.274024 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa847cdf-6cc3-45ec-a474-d4fcda263595 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:32:55.276038 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa847cdf-6cc3-45ec-a474-d4fcda263595 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:32:55.358062 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1357/2697] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:32:55 2022] POST /identity/v3/projects => generated 381 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:32:55.362768 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c576a2f-32b1-4760-b7bc-94b2464a1f16 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:32:55.373038 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c576a2f-32b1-4760-b7bc-94b2464a1f16 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:32:55.428289 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fefba9-0c5c-4bcc-9b19-f2a68718eca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:32:55.428900 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fefba9-0c5c-4bcc-9b19-f2a68718eca9 admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:55.429180 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fefba9-0c5c-4bcc-9b19-f2a68718eca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:55.429453 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fefba9-0c5c-4bcc-9b19-f2a68718eca9 admin admin] PATH_INFO: `/v3/users` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:55.430070 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fefba9-0c5c-4bcc-9b19-f2a68718eca9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:32:55.432010 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fefba9-0c5c-4bcc-9b19-f2a68718eca9 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:32:55.510213 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1341/2698] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:32:55 2022] POST /identity/v3/users => generated 291 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:32:55.514260 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa847cdf-6cc3-45ec-a474-d4fcda263595 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:32:55.524961 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa847cdf-6cc3-45ec-a474-d4fcda263595 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:32:55.529519 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12e15fd-50a2-4ca0-95b6-eb911b8a1d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:32:55.530113 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12e15fd-50a2-4ca0-95b6-eb911b8a1d20 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:55.530971 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12e15fd-50a2-4ca0-95b6-eb911b8a1d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:55.531250 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12e15fd-50a2-4ca0-95b6-eb911b8a1d20 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:55.531886 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12e15fd-50a2-4ca0-95b6-eb911b8a1d20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:32:55.533472 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12e15fd-50a2-4ca0-95b6-eb911b8a1d20 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:32:55.543653 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1358/2699] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:32:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:32:55.547630 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fefba9-0c5c-4bcc-9b19-f2a68718eca9 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:32:55.557461 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fefba9-0c5c-4bcc-9b19-f2a68718eca9 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:32:55.562324 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a04d7db-60f0-4d3c-85cb-696a1bbb29fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:32:55.562920 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a04d7db-60f0-4d3c-85cb-696a1bbb29fb admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:55.563207 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a04d7db-60f0-4d3c-85cb-696a1bbb29fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:55.563486 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a04d7db-60f0-4d3c-85cb-696a1bbb29fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:55.564127 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a04d7db-60f0-4d3c-85cb-696a1bbb29fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:32:55.565736 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a04d7db-60f0-4d3c-85cb-696a1bbb29fb admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:32:55.576109 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1342/2700] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:32:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:32:55.580795 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12e15fd-50a2-4ca0-95b6-eb911b8a1d20 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:32:55.592012 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12e15fd-50a2-4ca0-95b6-eb911b8a1d20 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:32:55.597442 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b5c903-016b-4df4-a0b1-8e4353797baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:32:55.598132 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b5c903-016b-4df4-a0b1-8e4353797baa admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:55.598445 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b5c903-016b-4df4-a0b1-8e4353797baa admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:55.598748 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b5c903-016b-4df4-a0b1-8e4353797baa admin admin] PATH_INFO: `/v3/projects/0b8107c0e697482ea144a7026a568ff3/users/43d7031613814c0eb40662b3b893bd49/roles/3255f77192524a16bc405173106871fc` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:55.601051 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:32:55.613837 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b5c903-016b-4df4-a0b1-8e4353797baa admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b8107c0e697482ea144a7026a568ff3, user_id=43d7031613814c0eb40662b3b893bd49, role_id=3255f77192524a16bc405173106871fc)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:32:55.616562 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b5c903-016b-4df4-a0b1-8e4353797baa admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:32:55.669975 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1359/2701] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:32:55 2022] PUT /identity/v3/projects/0b8107c0e697482ea144a7026a568ff3/users/43d7031613814c0eb40662b3b893bd49/roles/3255f77192524a16bc405173106871fc => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:32:55.675195 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e517292-3e12-4d6c-be85-401ae60d8292 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:55.675483 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e517292-3e12-4d6c-be85-401ae60d8292 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:55.675754 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e517292-3e12-4d6c-be85-401ae60d8292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:55.694561 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-3e517292-3e12-4d6c-be85-401ae60d8292 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:32:55.695751 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-3e517292-3e12-4d6c-be85-401ae60d8292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:32:55.754379 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1343/2702] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:32:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:32:55.764148 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b5c903-016b-4df4-a0b1-8e4353797baa admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:32:55.766366 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-83b5c903-016b-4df4-a0b1-8e4353797baa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:32:55.822656 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b5c903-016b-4df4-a0b1-8e4353797baa admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:32:55.826244 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b65cc1-403d-4dbf-b605-549d8216db3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:32:55.826873 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b65cc1-403d-4dbf-b605-549d8216db3e service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:55.827755 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b65cc1-403d-4dbf-b605-549d8216db3e service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:55.828036 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b65cc1-403d-4dbf-b605-549d8216db3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:55.836777 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b65cc1-403d-4dbf-b605-549d8216db3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:32:55.838200 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b65cc1-403d-4dbf-b605-549d8216db3e service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:32:55.847495 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1360/2703] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:32:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:32:56.957157 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4ee525-12e2-4c67-88e8-e49f690659f0 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:56.957723 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4ee525-12e2-4c67-88e8-e49f690659f0 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:56.957825 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4ee525-12e2-4c67-88e8-e49f690659f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:56.987524 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-5b4ee525-12e2-4c67-88e8-e49f690659f0 None None] MFA Rules not processed for user `43d7031613814c0eb40662b3b893bd49`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:32:56.989094 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-5b4ee525-12e2-4c67-88e8-e49f690659f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:32:57.026123 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:32:57.057006 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1344/2704] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:32:56 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:32:57.068766 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b65cc1-403d-4dbf-b605-549d8216db3e service neutron] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:32:57.070165 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-b5b65cc1-403d-4dbf-b605-549d8216db3e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:32:57.126363 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b65cc1-403d-4dbf-b605-549d8216db3e service neutron] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:32:57.129986 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f901f8-19ad-45de-9e70-8ea051b351d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60193ba9e1c14df3bc9eebcc6f28f2d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:32:57.130627 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f901f8-19ad-45de-9e70-8ea051b351d8 service cinder] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:57.130912 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f901f8-19ad-45de-9e70-8ea051b351d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:57.131190 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f901f8-19ad-45de-9e70-8ea051b351d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:57.139492 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f901f8-19ad-45de-9e70-8ea051b351d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:32:57.141479 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f901f8-19ad-45de-9e70-8ea051b351d8 service cinder] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:32:57.150201 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1361/2705] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:32:57 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:32:58.860558 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd41fb1c-20d3-4527-991c-9a41ac0c6470 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:58.861109 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd41fb1c-20d3-4527-991c-9a41ac0c6470 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:58.861109 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd41fb1c-20d3-4527-991c-9a41ac0c6470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:58.881435 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-cd41fb1c-20d3-4527-991c-9a41ac0c6470 None None] MFA Rules not processed for user `43d7031613814c0eb40662b3b893bd49`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:32:58.882853 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-cd41fb1c-20d3-4527-991c-9a41ac0c6470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:32:58.890320 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1345/2706] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:32:58 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:32:59.121981 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f901f8-19ad-45de-9e70-8ea051b351d8 service cinder] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:32:59.123453 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-83f901f8-19ad-45de-9e70-8ea051b351d8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:32:59.135933 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f901f8-19ad-45de-9e70-8ea051b351d8 service cinder] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:32:59.139523 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d3dcd8-a97a-4115-9605-0344fffa3d64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:32:59.140774 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d3dcd8-a97a-4115-9605-0344fffa3d64 service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:32:59.141005 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d3dcd8-a97a-4115-9605-0344fffa3d64 service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:32:59.141278 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d3dcd8-a97a-4115-9605-0344fffa3d64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:32:59.142964 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-b7d3dcd8-a97a-4115-9605-0344fffa3d64 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:32:59.152163 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d3dcd8-a97a-4115-9605-0344fffa3d64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:32:59.153572 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d3dcd8-a97a-4115-9605-0344fffa3d64 service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:32:59.161952 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1362/2707] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:32:59 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:33:08.876665 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=14 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:33:09.158087 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=65,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:34:04.802860 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd41fb1c-20d3-4527-991c-9a41ac0c6470 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:34:04.804263 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-cd41fb1c-20d3-4527-991c-9a41ac0c6470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:34:04.816612 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd41fb1c-20d3-4527-991c-9a41ac0c6470 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:34:04.820633 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67333409-d834-4770-8bc7-e641b891689f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60193ba9e1c14df3bc9eebcc6f28f2d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:34:04.822639 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67333409-d834-4770-8bc7-e641b891689f service cinder] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:34:04.822639 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67333409-d834-4770-8bc7-e641b891689f service cinder] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:34:04.822639 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67333409-d834-4770-8bc7-e641b891689f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:34:04.823627 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-67333409-d834-4770-8bc7-e641b891689f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:34:04.898861 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67333409-d834-4770-8bc7-e641b891689f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:34:04.898861 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67333409-d834-4770-8bc7-e641b891689f service cinder] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:34:04.907914 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1346/2708] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:34:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:34:14.902770 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=34 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:34:17.810858 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d3dcd8-a97a-4115-9605-0344fffa3d64 service neutron] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:34:17.819739 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d3dcd8-a97a-4115-9605-0344fffa3d64 service neutron] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:34:17.823457 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8d1442-ed3d-4e9a-a4c6-6c16ca00a1b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:34:17.824051 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8d1442-ed3d-4e9a-a4c6-6c16ca00a1b8 service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:34:17.824333 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8d1442-ed3d-4e9a-a4c6-6c16ca00a1b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:34:17.824607 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8d1442-ed3d-4e9a-a4c6-6c16ca00a1b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:34:17.826343 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-7b8d1442-ed3d-4e9a-a4c6-6c16ca00a1b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:34:17.840369 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8d1442-ed3d-4e9a-a4c6-6c16ca00a1b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:34:17.841814 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8d1442-ed3d-4e9a-a4c6-6c16ca00a1b8 service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:34:17.853451 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1363/2709] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:34:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:34:23.703526 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d89ffaf-feac-4a4c-967d-d994159f7aaa None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:34:23.703974 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d89ffaf-feac-4a4c-967d-d994159f7aaa None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:34:23.704086 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d89ffaf-feac-4a4c-967d-d994159f7aaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:34:23.723281 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-3d89ffaf-feac-4a4c-967d-d994159f7aaa None None] MFA Rules not processed for user `43d7031613814c0eb40662b3b893bd49`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:34:23.724475 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-3d89ffaf-feac-4a4c-967d-d994159f7aaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:34:23.732083 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1347/2710] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:34:23 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:34:27.847984 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=6 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:34:33.717728 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=3 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:34:34.457873 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8d1442-ed3d-4e9a-a4c6-6c16ca00a1b8 service neutron] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:34:34.466810 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8d1442-ed3d-4e9a-a4c6-6c16ca00a1b8 service neutron] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:34:34.470714 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c0af3a-f2c1-4457-bdb8-e3da79ae254d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:34:34.471341 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c0af3a-f2c1-4457-bdb8-e3da79ae254d service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:34:34.471627 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c0af3a-f2c1-4457-bdb8-e3da79ae254d service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:34:34.471902 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c0af3a-f2c1-4457-bdb8-e3da79ae254d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:34:34.473888 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-e4c0af3a-f2c1-4457-bdb8-e3da79ae254d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:34:34.484869 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c0af3a-f2c1-4457-bdb8-e3da79ae254d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:34:34.487013 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c0af3a-f2c1-4457-bdb8-e3da79ae254d service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:34:34.495520 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1364/2711] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:34:34 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:34:40.733770 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5add08e-f6ff-4e2c-b6ea-ca5bd1df4c44 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:34:40.734248 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5add08e-f6ff-4e2c-b6ea-ca5bd1df4c44 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:34:40.734380 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5add08e-f6ff-4e2c-b6ea-ca5bd1df4c44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:34:40.753242 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-e5add08e-f6ff-4e2c-b6ea-ca5bd1df4c44 None None] MFA Rules not processed for user `712c35b765b64919aa3eaaf02892590d`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:34:40.754513 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-e5add08e-f6ff-4e2c-b6ea-ca5bd1df4c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:34:40.809619 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1348/2712] 10.222.0.58 () {58 vars in 1009 bytes} [Thu Aug 25 03:34:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:34:40.924055 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c0af3a-f2c1-4457-bdb8-e3da79ae254d service neutron] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:34:40.925506 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-e4c0af3a-f2c1-4457-bdb8-e3da79ae254d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:34:40.993202 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c0af3a-f2c1-4457-bdb8-e3da79ae254d service neutron] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:34:40.998709 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8cc4d7-56aa-4dfb-aea3-89dcefee2deb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5047e1a7cd2d416d90b2d8c5a6596522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:34:40.999321 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8cc4d7-56aa-4dfb-aea3-89dcefee2deb service swift] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:34:40.999600 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8cc4d7-56aa-4dfb-aea3-89dcefee2deb service swift] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:34:40.999875 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8cc4d7-56aa-4dfb-aea3-89dcefee2deb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:34:41.007753 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8cc4d7-56aa-4dfb-aea3-89dcefee2deb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:34:41.009308 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8cc4d7-56aa-4dfb-aea3-89dcefee2deb service swift] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:34:41.018050 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1365/2713] 10.222.0.58 () {60 vars in 1384 bytes} [Thu Aug 25 03:34:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 25 03:34:50.802756 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=25 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:34:51.014822 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=38 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:35:01.826039 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8789dd-7841-4aed-b7e2-17ea3759cedb None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:35:01.826520 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8789dd-7841-4aed-b7e2-17ea3759cedb None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:35:01.826627 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8789dd-7841-4aed-b7e2-17ea3759cedb None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:35:01.828006 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1349/2714] 10.222.0.58 () {54 vars in 921 bytes} [Thu Aug 25 03:35:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:35:01.833857 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c2195-9b2f-429c-99c8-f49c977ac6b2 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:35:01.837104 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c2195-9b2f-429c-99c8-f49c977ac6b2 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:35:01.837104 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c2195-9b2f-429c-99c8-f49c977ac6b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:35:01.854686 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-9c8c2195-9b2f-429c-99c8-f49c977ac6b2 None None] MFA Rules not processed for user `0cd171bf4df94f8fab397e42835feeab`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:35:01.855902 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-9c8c2195-9b2f-429c-99c8-f49c977ac6b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:35:01.865841 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1366/2715] 10.222.0.58 () {58 vars in 1018 bytes} [Thu Aug 25 03:35:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:35:01.878404 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8789dd-7841-4aed-b7e2-17ea3759cedb None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:35:01.880097 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-3c8789dd-7841-4aed-b7e2-17ea3759cedb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:35:01.938550 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8789dd-7841-4aed-b7e2-17ea3759cedb None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:35:01.942041 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b1a2e5-3b1f-4cf6-91e5-d897a2990e1e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f89089b390c4b6599e54000c0fa082a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:35:01.942693 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b1a2e5-3b1f-4cf6-91e5-d897a2990e1e service glance] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:35:01.942996 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b1a2e5-3b1f-4cf6-91e5-d897a2990e1e service glance] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:35:01.943265 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b1a2e5-3b1f-4cf6-91e5-d897a2990e1e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:35:01.955104 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b1a2e5-3b1f-4cf6-91e5-d897a2990e1e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:35:01.956522 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b1a2e5-3b1f-4cf6-91e5-d897a2990e1e service glance] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:35:01.966985 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1350/2716] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:35:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:35:02.185338 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa00fa-e398-4112-b61b-425dc843c4ab None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:35:02.185651 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa00fa-e398-4112-b61b-425dc843c4ab None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:35:02.185936 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa00fa-e398-4112-b61b-425dc843c4ab None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:35:02.187360 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1367/2717] 10.222.0.58 () {54 vars in 921 bytes} [Thu Aug 25 03:35:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:35:02.192342 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e77999-25de-43e2-8629-09ba6b8a2ea0 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:35:02.192632 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e77999-25de-43e2-8629-09ba6b8a2ea0 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:35:02.192993 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e77999-25de-43e2-8629-09ba6b8a2ea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:35:02.213477 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-41e77999-25de-43e2-8629-09ba6b8a2ea0 None None] MFA Rules not processed for user `a0a1a7f091774842907fd08de15e8058`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:35:02.215488 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-41e77999-25de-43e2-8629-09ba6b8a2ea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:35:02.221132 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1351/2718] 10.222.0.58 () {58 vars in 1018 bytes} [Thu Aug 25 03:35:02 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:35:02.233310 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aa00fa-e398-4112-b61b-425dc843c4ab None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:35:02.244648 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aa00fa-e398-4112-b61b-425dc843c4ab None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:35:02.248153 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb3ba6f-7895-47f4-8be4-bf8c6a67d5c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:35:02.248748 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb3ba6f-7895-47f4-8be4-bf8c6a67d5c2 service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:35:02.249032 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb3ba6f-7895-47f4-8be4-bf8c6a67d5c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:35:02.249302 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb3ba6f-7895-47f4-8be4-bf8c6a67d5c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:35:02.256194 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb3ba6f-7895-47f4-8be4-bf8c6a67d5c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:35:02.257610 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb3ba6f-7895-47f4-8be4-bf8c6a67d5c2 service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:35:02.268044 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1368/2719] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:35:02 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:35:02.351354 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e9085f-e999-468c-bf67-e4b33307632c None None] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:35:02.352259 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e9085f-e999-468c-bf67-e4b33307632c None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:35:02.352614 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e9085f-e999-468c-bf67-e4b33307632c None None] PATH_INFO: `/` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:35:02.354804 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1352/2720] 10.222.0.58 () {54 vars in 921 bytes} [Thu Aug 25 03:35:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:35:02.360801 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4864a49f-ba1f-4fb6-8e2f-00a861d9e21e None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:35:02.369664 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4864a49f-ba1f-4fb6-8e2f-00a861d9e21e None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:35:02.369664 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4864a49f-ba1f-4fb6-8e2f-00a861d9e21e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:35:02.381210 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-4864a49f-ba1f-4fb6-8e2f-00a861d9e21e None None] MFA Rules not processed for user `0cd171bf4df94f8fab397e42835feeab`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:35:02.382453 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-4864a49f-ba1f-4fb6-8e2f-00a861d9e21e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:35:02.392055 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1369/2721] 10.222.0.58 () {58 vars in 1018 bytes} [Thu Aug 25 03:35:02 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:35:02.402643 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e9085f-e999-468c-bf67-e4b33307632c None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:35:02.413878 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e9085f-e999-468c-bf67-e4b33307632c None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:35:02.418210 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6cf4ff-d82d-4255-bc71-249a6ecc1050 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:35:02.418839 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6cf4ff-d82d-4255-bc71-249a6ecc1050 service neutron] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:35:02.419118 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6cf4ff-d82d-4255-bc71-249a6ecc1050 service neutron] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:35:02.419391 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6cf4ff-d82d-4255-bc71-249a6ecc1050 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:35:02.429487 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6cf4ff-d82d-4255-bc71-249a6ecc1050 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:35:02.430941 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6cf4ff-d82d-4255-bc71-249a6ecc1050 service neutron] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:35:02.442944 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1353/2722] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:35:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:35:02.506086 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4864a49f-ba1f-4fb6-8e2f-00a861d9e21e None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:35:02.507581 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-4864a49f-ba1f-4fb6-8e2f-00a861d9e21e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:35:02.574913 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4864a49f-ba1f-4fb6-8e2f-00a861d9e21e None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:35:02.580171 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adaeed8-0b56-4b0f-9e3a-7eff053c82a3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b692edd3e8c41289cc42e1afb4a6dd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:35:02.580780 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adaeed8-0b56-4b0f-9e3a-7eff053c82a3 service placement] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:35:02.581057 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adaeed8-0b56-4b0f-9e3a-7eff053c82a3 service placement] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:35:02.581376 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adaeed8-0b56-4b0f-9e3a-7eff053c82a3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:35:02.583190 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-2adaeed8-0b56-4b0f-9e3a-7eff053c82a3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:35:02.597907 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adaeed8-0b56-4b0f-9e3a-7eff053c82a3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:35:02.599589 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adaeed8-0b56-4b0f-9e3a-7eff053c82a3 service placement] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:35:02.610252 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1370/2723] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:35:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:35:06.565794 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b877cbe1-3461-4761-8b57-dd41c581513a None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:35:06.566300 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b877cbe1-3461-4761-8b57-dd41c581513a None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:35:06.566403 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b877cbe1-3461-4761-8b57-dd41c581513a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:35:06.586536 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-b877cbe1-3461-4761-8b57-dd41c581513a None None] MFA Rules not processed for user `712c35b765b64919aa3eaaf02892590d`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:35:06.587890 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-b877cbe1-3461-4761-8b57-dd41c581513a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:35:06.595515 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1354/2724] 10.222.0.58 () {58 vars in 1009 bytes} [Thu Aug 25 03:35:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:35:06.607566 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adaeed8-0b56-4b0f-9e3a-7eff053c82a3 service placement] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:35:06.619182 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adaeed8-0b56-4b0f-9e3a-7eff053c82a3 service placement] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:35:06.624276 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25488368-ab45-46ad-9a42-38bb60fb6656 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5047e1a7cd2d416d90b2d8c5a6596522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:35:06.624912 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25488368-ab45-46ad-9a42-38bb60fb6656 service swift] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:35:06.626091 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25488368-ab45-46ad-9a42-38bb60fb6656 service swift] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:35:06.626091 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25488368-ab45-46ad-9a42-38bb60fb6656 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:35:06.635020 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25488368-ab45-46ad-9a42-38bb60fb6656 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:35:06.637205 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25488368-ab45-46ad-9a42-38bb60fb6656 service swift] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:35:06.645365 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1371/2725] 10.222.0.58 () {60 vars in 1384 bytes} [Thu Aug 25 03:35:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 25 03:35:11.287730 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b877cbe1-3461-4761-8b57-dd41c581513a None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:35:11.289300 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-b877cbe1-3461-4761-8b57-dd41c581513a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:35:11.309501 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b877cbe1-3461-4761-8b57-dd41c581513a None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:35:11.315463 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86682a09-a354-4e6d-9235-b295add9e523 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cd171bf4df94f8fab397e42835feeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:35:11.316113 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86682a09-a354-4e6d-9235-b295add9e523 service nova] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:35:11.316394 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86682a09-a354-4e6d-9235-b295add9e523 service nova] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:35:11.316736 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86682a09-a354-4e6d-9235-b295add9e523 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:35:11.322511 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-86682a09-a354-4e6d-9235-b295add9e523 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:35:11.336084 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86682a09-a354-4e6d-9235-b295add9e523 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:35:11.338307 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86682a09-a354-4e6d-9235-b295add9e523 service nova] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:35:11.366727 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1355/2726] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:35:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:35:16.641924 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=50 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:35:18.857488 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25488368-ab45-46ad-9a42-38bb60fb6656 service swift] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:35:18.870016 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25488368-ab45-46ad-9a42-38bb60fb6656 service swift] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:35:18.875531 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f96114e-cf94-44e4-8602-f299bfc45ab5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cd171bf4df94f8fab397e42835feeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:35:18.876125 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f96114e-cf94-44e4-8602-f299bfc45ab5 service nova] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:35:18.876408 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f96114e-cf94-44e4-8602-f299bfc45ab5 service nova] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:35:18.876686 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f96114e-cf94-44e4-8602-f299bfc45ab5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:35:18.878419 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-3f96114e-cf94-44e4-8602-f299bfc45ab5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:35:18.890524 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f96114e-cf94-44e4-8602-f299bfc45ab5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:35:18.897968 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f96114e-cf94-44e4-8602-f299bfc45ab5 service nova] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:35:18.902650 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1372/2727] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:35:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:35:21.346151 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=44 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:35:28.897976 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=6 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:35:29.207762 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b52d69-fe20-46e5-bd9c-044b27ff6b20 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:35:29.208265 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b52d69-fe20-46e5-bd9c-044b27ff6b20 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:35:29.208428 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b52d69-fe20-46e5-bd9c-044b27ff6b20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:35:29.230073 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-12b52d69-fe20-46e5-bd9c-044b27ff6b20 None None] MFA Rules not processed for user `712c35b765b64919aa3eaaf02892590d`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:35:29.231331 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-12b52d69-fe20-46e5-bd9c-044b27ff6b20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:35:29.244061 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1356/2728] 10.222.0.58 () {58 vars in 967 bytes} [Thu Aug 25 03:35:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:35:29.253866 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f96114e-cf94-44e4-8602-f299bfc45ab5 service nova] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:35:29.266549 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f96114e-cf94-44e4-8602-f299bfc45ab5 service nova] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:35:29.273197 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dd7a91-27c9-4f3b-b107-63b43535c17a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5047e1a7cd2d416d90b2d8c5a6596522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:35:29.273197 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dd7a91-27c9-4f3b-b107-63b43535c17a service swift] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:35:29.273197 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dd7a91-27c9-4f3b-b107-63b43535c17a service swift] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:35:29.274125 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dd7a91-27c9-4f3b-b107-63b43535c17a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:35:29.283563 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dd7a91-27c9-4f3b-b107-63b43535c17a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:35:29.285205 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dd7a91-27c9-4f3b-b107-63b43535c17a service swift] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:35:29.294600 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1373/2729] 10.222.0.58 () {60 vars in 1384 bytes} [Thu Aug 25 03:35:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 25 03:35:39.223936 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=3 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:35:39.289999 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=6 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:35:49.637304 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b52d69-fe20-46e5-bd9c-044b27ff6b20 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:35:49.650262 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b52d69-fe20-46e5-bd9c-044b27ff6b20 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:35:49.654715 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8277ed2-2368-435c-be8a-53173f4f1e44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:35:49.656055 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8277ed2-2368-435c-be8a-53173f4f1e44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:35:49.656338 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8277ed2-2368-435c-be8a-53173f4f1e44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:35:49.656651 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8277ed2-2368-435c-be8a-53173f4f1e44 admin admin] PATH_INFO: `/v3/users/43d7031613814c0eb40662b3b893bd49` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:35:49.657731 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8277ed2-2368-435c-be8a-53173f4f1e44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=43d7031613814c0eb40662b3b893bd49)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:35:49.659695 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8277ed2-2368-435c-be8a-53173f4f1e44 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:35:49.785593 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-d8277ed2-2368-435c-be8a-53173f4f1e44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43d7031613814c0eb40662b3b893bd49'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:35:49.845607 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-d8277ed2-2368-435c-be8a-53173f4f1e44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43d7031613814c0eb40662b3b893bd49'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:35:49.846378 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-d8277ed2-2368-435c-be8a-53173f4f1e44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43d7031613814c0eb40662b3b893bd49'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:35:49.846992 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-d8277ed2-2368-435c-be8a-53173f4f1e44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43d7031613814c0eb40662b3b893bd49'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:35:49.860752 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-d8277ed2-2368-435c-be8a-53173f4f1e44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43d7031613814c0eb40662b3b893bd49'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:35:49.873527 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1357/2730] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:35:49 2022] DELETE /identity/v3/users/43d7031613814c0eb40662b3b893bd49 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:35:50.087490 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dd7a91-27c9-4f3b-b107-63b43535c17a service swift] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:35:50.098883 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dd7a91-27c9-4f3b-b107-63b43535c17a service swift] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:35:50.154397 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bc2a5c-54b2-40d6-9491-c89a5a9dbca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:35:50.155024 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bc2a5c-54b2-40d6-9491-c89a5a9dbca3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:35:50.155303 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bc2a5c-54b2-40d6-9491-c89a5a9dbca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:35:50.155576 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bc2a5c-54b2-40d6-9491-c89a5a9dbca3 admin admin] PATH_INFO: `/v3/projects/0b8107c0e697482ea144a7026a568ff3` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:35:50.157308 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bc2a5c-54b2-40d6-9491-c89a5a9dbca3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b8107c0e697482ea144a7026a568ff3)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:35:50.159303 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bc2a5c-54b2-40d6-9491-c89a5a9dbca3 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:35:50.268001 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-72bc2a5c-54b2-40d6-9491-c89a5a9dbca3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b8107c0e697482ea144a7026a568ff3', 'request_id': 'req-72bc2a5c-54b2-40d6-9491-c89a5a9dbca3'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:35:50.319278 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-72bc2a5c-54b2-40d6-9491-c89a5a9dbca3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b8107c0e697482ea144a7026a568ff3', 'request_id': 'req-72bc2a5c-54b2-40d6-9491-c89a5a9dbca3'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:35:50.324927 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-72bc2a5c-54b2-40d6-9491-c89a5a9dbca3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b8107c0e697482ea144a7026a568ff3', 'request_id': 'req-72bc2a5c-54b2-40d6-9491-c89a5a9dbca3'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:35:50.340286 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-72bc2a5c-54b2-40d6-9491-c89a5a9dbca3 admin admin] The token cache is being invalidate because project 0b8107c0e697482ea144a7026a568ff3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:35:50.340586 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-72bc2a5c-54b2-40d6-9491-c89a5a9dbca3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:35:50.341616 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-72bc2a5c-54b2-40d6-9491-c89a5a9dbca3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:35:50.343131 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1374/2731] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:35:50 2022] DELETE /identity/v3/projects/0b8107c0e697482ea144a7026a568ff3 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:35:50.350133 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b73ce7-0eb3-491f-8fcc-349181b65cfc None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:35:50.350482 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b73ce7-0eb3-491f-8fcc-349181b65cfc None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:35:50.350771 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b73ce7-0eb3-491f-8fcc-349181b65cfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:35:50.370256 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-75b73ce7-0eb3-491f-8fcc-349181b65cfc None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:35:50.371497 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-75b73ce7-0eb3-491f-8fcc-349181b65cfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:35:50.428878 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1358/2732] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:35:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:35:50.436565 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e8de70-31d9-4b16-952a-4c22eef2e1e0 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:35:50.436853 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e8de70-31d9-4b16-952a-4c22eef2e1e0 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:35:50.437131 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e8de70-31d9-4b16-952a-4c22eef2e1e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:35:50.458231 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-f5e8de70-31d9-4b16-952a-4c22eef2e1e0 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:35:50.459452 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-f5e8de70-31d9-4b16-952a-4c22eef2e1e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:35:50.467202 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1375/2733] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:35:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:35:50.471209 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b73ce7-0eb3-491f-8fcc-349181b65cfc None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:35:50.481123 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b73ce7-0eb3-491f-8fcc-349181b65cfc None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:35:50.485715 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d814e6-ad74-4056-9a28-e0050530a6a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:35:50.486359 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d814e6-ad74-4056-9a28-e0050530a6a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:35:50.486638 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d814e6-ad74-4056-9a28-e0050530a6a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:35:50.486911 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d814e6-ad74-4056-9a28-e0050530a6a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:35:50.487552 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d814e6-ad74-4056-9a28-e0050530a6a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:35:50.487852 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d814e6-ad74-4056-9a28-e0050530a6a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:35:50.489428 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d814e6-ad74-4056-9a28-e0050530a6a1 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:35:50.501132 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1359/2734] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Aug 25 03:35:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:35:50.505503 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e8de70-31d9-4b16-952a-4c22eef2e1e0 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:35:50.514773 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e8de70-31d9-4b16-952a-4c22eef2e1e0 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:35:50.520128 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84eaed1-fce3-4282-b94b-09f2b2dbc309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:35:50.520943 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84eaed1-fce3-4282-b94b-09f2b2dbc309 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:35:50.521374 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84eaed1-fce3-4282-b94b-09f2b2dbc309 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:35:50.521825 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84eaed1-fce3-4282-b94b-09f2b2dbc309 admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:35:50.522896 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84eaed1-fce3-4282-b94b-09f2b2dbc309 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:35:50.525660 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84eaed1-fce3-4282-b94b-09f2b2dbc309 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:35:50.612607 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1376/2735] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:35:50 2022] POST /identity/v3/projects => generated 371 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:35:50.616729 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d814e6-ad74-4056-9a28-e0050530a6a1 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:35:50.628444 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d814e6-ad74-4056-9a28-e0050530a6a1 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:35:50.684475 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bbf4f2-8ad1-4c76-a662-472cad130304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:35:50.685132 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bbf4f2-8ad1-4c76-a662-472cad130304 admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:35:50.685344 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bbf4f2-8ad1-4c76-a662-472cad130304 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:35:50.685611 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bbf4f2-8ad1-4c76-a662-472cad130304 admin admin] PATH_INFO: `/v3/users` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:35:50.686218 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bbf4f2-8ad1-4c76-a662-472cad130304 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:35:50.688689 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bbf4f2-8ad1-4c76-a662-472cad130304 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:35:50.696355 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:35:50.764939 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1360/2736] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:35:50 2022] POST /identity/v3/users => generated 286 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:35:50.769654 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84eaed1-fce3-4282-b94b-09f2b2dbc309 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:35:50.780045 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84eaed1-fce3-4282-b94b-09f2b2dbc309 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:35:50.784648 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6da34d-1d9a-4878-8fc5-ab5565ad2a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:35:50.785236 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6da34d-1d9a-4878-8fc5-ab5565ad2a03 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:35:50.785513 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6da34d-1d9a-4878-8fc5-ab5565ad2a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:35:50.785783 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6da34d-1d9a-4878-8fc5-ab5565ad2a03 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:35:50.786487 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6da34d-1d9a-4878-8fc5-ab5565ad2a03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:35:50.788068 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6da34d-1d9a-4878-8fc5-ab5565ad2a03 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:35:50.799042 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1377/2737] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:35:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:35:50.807506 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bbf4f2-8ad1-4c76-a662-472cad130304 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:35:50.818189 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bbf4f2-8ad1-4c76-a662-472cad130304 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:35:50.823894 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549451fb-a51b-4d30-becc-03598f51e2e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:35:50.824489 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549451fb-a51b-4d30-becc-03598f51e2e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:35:50.824769 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549451fb-a51b-4d30-becc-03598f51e2e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:35:50.825109 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549451fb-a51b-4d30-becc-03598f51e2e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:35:50.825753 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549451fb-a51b-4d30-becc-03598f51e2e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:35:50.827393 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549451fb-a51b-4d30-becc-03598f51e2e2 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:35:50.838040 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1361/2738] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:35:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:35:50.841850 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6da34d-1d9a-4878-8fc5-ab5565ad2a03 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:35:50.851822 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6da34d-1d9a-4878-8fc5-ab5565ad2a03 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:35:50.856455 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a32dec5-497c-4e67-8398-ba50c8359c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:35:50.857175 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a32dec5-497c-4e67-8398-ba50c8359c61 admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:35:50.857461 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a32dec5-497c-4e67-8398-ba50c8359c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:35:50.857734 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a32dec5-497c-4e67-8398-ba50c8359c61 admin admin] PATH_INFO: `/v3/projects/d34a968ff59e43e796b4b48b35374bc7/users/9f32ba4c897c4003ba27ebc78c6d05a1/roles/3255f77192524a16bc405173106871fc` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:35:50.872466 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a32dec5-497c-4e67-8398-ba50c8359c61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d34a968ff59e43e796b4b48b35374bc7, user_id=9f32ba4c897c4003ba27ebc78c6d05a1, role_id=3255f77192524a16bc405173106871fc)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:35:50.875710 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a32dec5-497c-4e67-8398-ba50c8359c61 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:35:50.942465 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1378/2739] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:35:50 2022] PUT /identity/v3/projects/d34a968ff59e43e796b4b48b35374bc7/users/9f32ba4c897c4003ba27ebc78c6d05a1/roles/3255f77192524a16bc405173106871fc => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:35:50.947587 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1c2d18-48d3-4918-b3ae-d687f561f6ad None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:35:50.953332 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1c2d18-48d3-4918-b3ae-d687f561f6ad None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:35:50.953332 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1c2d18-48d3-4918-b3ae-d687f561f6ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:35:50.966786 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-2d1c2d18-48d3-4918-b3ae-d687f561f6ad None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:35:50.967989 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-2d1c2d18-48d3-4918-b3ae-d687f561f6ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:35:51.027433 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1362/2740] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:35:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:35:51.037568 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a32dec5-497c-4e67-8398-ba50c8359c61 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:35:51.039321 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-1a32dec5-497c-4e67-8398-ba50c8359c61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:35:51.069573 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:35:51.095750 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a32dec5-497c-4e67-8398-ba50c8359c61 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:35:51.100570 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08518775-6ef2-4abb-a486-42070d1e28e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:35:51.101619 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08518775-6ef2-4abb-a486-42070d1e28e0 service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:35:51.101932 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08518775-6ef2-4abb-a486-42070d1e28e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:35:51.102216 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08518775-6ef2-4abb-a486-42070d1e28e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:35:51.121808 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08518775-6ef2-4abb-a486-42070d1e28e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:35:51.123387 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08518775-6ef2-4abb-a486-42070d1e28e0 service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:35:51.132792 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1379/2741] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:35:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:35:56.301447 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc575b6-a386-462c-ad8f-973067e4dee5 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:35:56.302018 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc575b6-a386-462c-ad8f-973067e4dee5 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:35:56.302018 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc575b6-a386-462c-ad8f-973067e4dee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:35:56.334860 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-1dc575b6-a386-462c-ad8f-973067e4dee5 None None] MFA Rules not processed for user `9f32ba4c897c4003ba27ebc78c6d05a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:35:56.336195 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-1dc575b6-a386-462c-ad8f-973067e4dee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:35:56.413346 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1363/2742] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:35:56 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:35:56.417934 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08518775-6ef2-4abb-a486-42070d1e28e0 service neutron] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:35:56.428531 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08518775-6ef2-4abb-a486-42070d1e28e0 service neutron] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:35:56.435852 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6fa40a-228e-423d-8003-ffb983d27c5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:35:56.439963 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6fa40a-228e-423d-8003-ffb983d27c5c admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:35:56.439963 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6fa40a-228e-423d-8003-ffb983d27c5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:35:56.439963 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6fa40a-228e-423d-8003-ffb983d27c5c admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:35:56.439963 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6fa40a-228e-423d-8003-ffb983d27c5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:35:56.441197 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6fa40a-228e-423d-8003-ffb983d27c5c admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:35:56.549921 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1380/2743] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:35:56 2022] POST /identity/v3/projects => generated 373 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:35:56.554040 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc575b6-a386-462c-ad8f-973067e4dee5 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:35:56.568944 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc575b6-a386-462c-ad8f-973067e4dee5 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:35:56.632159 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3801286c-60da-4e7d-bdb4-7ef146b46c88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:35:56.632159 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3801286c-60da-4e7d-bdb4-7ef146b46c88 admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:35:56.632159 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3801286c-60da-4e7d-bdb4-7ef146b46c88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:35:56.632159 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3801286c-60da-4e7d-bdb4-7ef146b46c88 admin admin] PATH_INFO: `/v3/users` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:35:56.632694 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3801286c-60da-4e7d-bdb4-7ef146b46c88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:35:56.633425 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3801286c-60da-4e7d-bdb4-7ef146b46c88 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:35:56.731557 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:35:56.751637 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1364/2744] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:35:56 2022] POST /identity/v3/users => generated 293 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:35:56.753109 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6fa40a-228e-423d-8003-ffb983d27c5c admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:35:56.766666 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6fa40a-228e-423d-8003-ffb983d27c5c admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:35:56.771785 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071a5197-9ba0-47bc-a3ba-25930b2a5efc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:35:56.772440 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071a5197-9ba0-47bc-a3ba-25930b2a5efc admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:35:56.772765 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071a5197-9ba0-47bc-a3ba-25930b2a5efc admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:35:56.773074 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071a5197-9ba0-47bc-a3ba-25930b2a5efc admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:35:56.773761 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071a5197-9ba0-47bc-a3ba-25930b2a5efc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:35:56.775844 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071a5197-9ba0-47bc-a3ba-25930b2a5efc admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:35:56.790837 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1381/2745] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:35:56 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 25 03:35:56.795831 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3801286c-60da-4e7d-bdb4-7ef146b46c88 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:35:56.806956 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3801286c-60da-4e7d-bdb4-7ef146b46c88 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:35:56.812174 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3861124a-e56b-40f9-8644-c4bc096c3555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:35:56.813464 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3861124a-e56b-40f9-8644-c4bc096c3555 admin admin] REQUEST_METHOD: `PUT` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:35:56.813814 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3861124a-e56b-40f9-8644-c4bc096c3555 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:35:56.814101 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3861124a-e56b-40f9-8644-c4bc096c3555 admin admin] PATH_INFO: `/v3/projects/51a8b420242541a08fce3ca9da4e8559/users/05c23669d16a4f6fa276cf1997dc49f4/roles/62f19ea272b447ccaab1576cd9bf4fd9` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:35:56.830106 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3861124a-e56b-40f9-8644-c4bc096c3555 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51a8b420242541a08fce3ca9da4e8559, user_id=05c23669d16a4f6fa276cf1997dc49f4, role_id=62f19ea272b447ccaab1576cd9bf4fd9)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:35:56.832818 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3861124a-e56b-40f9-8644-c4bc096c3555 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:35:56.932186 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1365/2746] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:35:56 2022] PUT /identity/v3/projects/51a8b420242541a08fce3ca9da4e8559/users/05c23669d16a4f6fa276cf1997dc49f4/roles/62f19ea272b447ccaab1576cd9bf4fd9 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:35:56.936036 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071a5197-9ba0-47bc-a3ba-25930b2a5efc admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:35:56.947929 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071a5197-9ba0-47bc-a3ba-25930b2a5efc admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:35:57.004606 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea6afeb-b4c1-489a-8828-4badffb5319c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:35:57.005234 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea6afeb-b4c1-489a-8828-4badffb5319c admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:35:57.005516 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea6afeb-b4c1-489a-8828-4badffb5319c admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:35:57.005793 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea6afeb-b4c1-489a-8828-4badffb5319c admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:35:57.006526 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea6afeb-b4c1-489a-8828-4badffb5319c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:35:57.008155 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea6afeb-b4c1-489a-8828-4badffb5319c admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:35:57.018839 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1382/2747] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:35:56 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 25 03:35:57.022930 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3861124a-e56b-40f9-8644-c4bc096c3555 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:35:57.036392 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3861124a-e56b-40f9-8644-c4bc096c3555 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:35:57.041418 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809e6b14-961d-4696-8f5b-3bab9819065b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:35:57.042090 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809e6b14-961d-4696-8f5b-3bab9819065b admin admin] REQUEST_METHOD: `PUT` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:35:57.042423 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809e6b14-961d-4696-8f5b-3bab9819065b admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:35:57.042703 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809e6b14-961d-4696-8f5b-3bab9819065b admin admin] PATH_INFO: `/v3/projects/51a8b420242541a08fce3ca9da4e8559/users/05c23669d16a4f6fa276cf1997dc49f4/roles/62f19ea272b447ccaab1576cd9bf4fd9` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:35:57.044729 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809e6b14-961d-4696-8f5b-3bab9819065b admin admin] RBAC: Authorizing `identity:create_grant(project_id=51a8b420242541a08fce3ca9da4e8559, user_id=05c23669d16a4f6fa276cf1997dc49f4, role_id=62f19ea272b447ccaab1576cd9bf4fd9)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:35:57.047963 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809e6b14-961d-4696-8f5b-3bab9819065b admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:35:57.059050 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1366/2748] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:35:57 2022] PUT /identity/v3/projects/51a8b420242541a08fce3ca9da4e8559/users/05c23669d16a4f6fa276cf1997dc49f4/roles/62f19ea272b447ccaab1576cd9bf4fd9 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:36:02.231407 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f676b4df-d043-4af4-8cf2-70331fbd475b None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:36:02.231948 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f676b4df-d043-4af4-8cf2-70331fbd475b None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:36:02.231948 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f676b4df-d043-4af4-8cf2-70331fbd475b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:36:02.270635 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-f676b4df-d043-4af4-8cf2-70331fbd475b None None] MFA Rules not processed for user `05c23669d16a4f6fa276cf1997dc49f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:36:02.271878 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-f676b4df-d043-4af4-8cf2-70331fbd475b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:36:02.348228 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1383/2749] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:36:02 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:36:02.369913 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809e6b14-961d-4696-8f5b-3bab9819065b admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:36:02.370853 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-809e6b14-961d-4696-8f5b-3bab9819065b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:36:02.509275 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809e6b14-961d-4696-8f5b-3bab9819065b admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:36:02.515657 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1363ab8e-0b98-42c4-a8ad-0e52fa94a906 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cd171bf4df94f8fab397e42835feeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:36:02.516298 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1363ab8e-0b98-42c4-a8ad-0e52fa94a906 service nova] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:36:02.516581 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1363ab8e-0b98-42c4-a8ad-0e52fa94a906 service nova] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:36:02.516859 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1363ab8e-0b98-42c4-a8ad-0e52fa94a906 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:36:02.588521 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1363ab8e-0b98-42c4-a8ad-0e52fa94a906 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:36:02.590325 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1363ab8e-0b98-42c4-a8ad-0e52fa94a906 service nova] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:36:02.601226 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1367/2750] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:36:02 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:36:05.020491 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0606ca-54cb-48bc-b6c3-f71e668553ea None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:36:05.021051 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0606ca-54cb-48bc-b6c3-f71e668553ea None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:36:05.021051 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0606ca-54cb-48bc-b6c3-f71e668553ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:36:05.041012 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-5a0606ca-54cb-48bc-b6c3-f71e668553ea None None] MFA Rules not processed for user `9f32ba4c897c4003ba27ebc78c6d05a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:36:05.042261 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-5a0606ca-54cb-48bc-b6c3-f71e668553ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:36:05.049891 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1384/2751] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:36:05 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:36:12.595963 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=72,keystone:INSERT=2 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:36:15.035822 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=94,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:36:27.182272 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1363ab8e-0b98-42c4-a8ad-0e52fa94a906 service nova] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:36:27.183739 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-1363ab8e-0b98-42c4-a8ad-0e52fa94a906 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:36:27.239713 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1363ab8e-0b98-42c4-a8ad-0e52fa94a906 service nova] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:36:27.243349 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cd7891-c5c9-43ce-8b5a-ef6c4bf7fa3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:36:27.243954 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cd7891-c5c9-43ce-8b5a-ef6c4bf7fa3a service neutron] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:36:27.244238 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cd7891-c5c9-43ce-8b5a-ef6c4bf7fa3a service neutron] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:36:27.244516 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cd7891-c5c9-43ce-8b5a-ef6c4bf7fa3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:36:27.246878 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-02cd7891-c5c9-43ce-8b5a-ef6c4bf7fa3a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:36:27.256679 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cd7891-c5c9-43ce-8b5a-ef6c4bf7fa3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:36:27.258106 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cd7891-c5c9-43ce-8b5a-ef6c4bf7fa3a service neutron] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:36:27.266038 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1368/2752] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:36:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:36:37.262408 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=26 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:36:44.046977 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0606ca-54cb-48bc-b6c3-f71e668553ea None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:36:44.057888 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0606ca-54cb-48bc-b6c3-f71e668553ea None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:36:44.061558 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6fa2c3-9870-436e-a692-139ae8595d01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:36:44.062192 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6fa2c3-9870-436e-a692-139ae8595d01 service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:36:44.062510 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6fa2c3-9870-436e-a692-139ae8595d01 service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:36:44.062789 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6fa2c3-9870-436e-a692-139ae8595d01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:36:44.064561 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-1e6fa2c3-9870-436e-a692-139ae8595d01 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:36:44.127128 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6fa2c3-9870-436e-a692-139ae8595d01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:36:44.128608 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6fa2c3-9870-436e-a692-139ae8595d01 service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:36:44.139008 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1385/2753] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:36:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:36:47.528287 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cd7891-c5c9-43ce-8b5a-ef6c4bf7fa3a service neutron] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:36:47.538909 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cd7891-c5c9-43ce-8b5a-ef6c4bf7fa3a service neutron] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:36:47.543217 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cebc32-83c1-460b-94c9-46e14448c7b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:36:47.544170 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cebc32-83c1-460b-94c9-46e14448c7b8 service neutron] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:36:47.544872 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cebc32-83c1-460b-94c9-46e14448c7b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:36:47.545387 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cebc32-83c1-460b-94c9-46e14448c7b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:36:47.558180 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cebc32-83c1-460b-94c9-46e14448c7b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:36:47.560026 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cebc32-83c1-460b-94c9-46e14448c7b8 service neutron] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:36:47.574638 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1369/2754] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:36:47 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:36:47.925451 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6fa2c3-9870-436e-a692-139ae8595d01 service neutron] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:36:47.927336 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-1e6fa2c3-9870-436e-a692-139ae8595d01 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:36:47.940434 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6fa2c3-9870-436e-a692-139ae8595d01 service neutron] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:36:47.951674 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a518aafa-184e-4d46-8381-f6b649fa7cfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:36:47.952423 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a518aafa-184e-4d46-8381-f6b649fa7cfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:36:47.952762 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a518aafa-184e-4d46-8381-f6b649fa7cfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:36:47.953125 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a518aafa-184e-4d46-8381-f6b649fa7cfb admin admin] PATH_INFO: `/v3/users/08a52103c0f4467c8366bd88e8b655e9` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:36:47.954901 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a518aafa-184e-4d46-8381-f6b649fa7cfb admin admin] RBAC: Authorizing `identity:delete_user(user_id=08a52103c0f4467c8366bd88e8b655e9)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:36:47.956834 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a518aafa-184e-4d46-8381-f6b649fa7cfb admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:36:48.173059 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-a518aafa-184e-4d46-8381-f6b649fa7cfb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08a52103c0f4467c8366bd88e8b655e9'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:36:48.262077 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-a518aafa-184e-4d46-8381-f6b649fa7cfb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08a52103c0f4467c8366bd88e8b655e9'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:36:48.262893 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-a518aafa-184e-4d46-8381-f6b649fa7cfb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08a52103c0f4467c8366bd88e8b655e9'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:36:48.271641 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-a518aafa-184e-4d46-8381-f6b649fa7cfb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08a52103c0f4467c8366bd88e8b655e9'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:36:48.284187 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-a518aafa-184e-4d46-8381-f6b649fa7cfb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08a52103c0f4467c8366bd88e8b655e9'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:36:48.290767 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1386/2755] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:36:47 2022] DELETE /identity/v3/users/08a52103c0f4467c8366bd88e8b655e9 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:36:48.549853 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cebc32-83c1-460b-94c9-46e14448c7b8 service neutron] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:36:48.560969 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cebc32-83c1-460b-94c9-46e14448c7b8 service neutron] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:36:48.641102 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1c6f60-e6b6-4e62-ae10-cdb0b94661ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:36:48.641708 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1c6f60-e6b6-4e62-ae10-cdb0b94661ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:36:48.642187 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1c6f60-e6b6-4e62-ae10-cdb0b94661ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:36:48.642496 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1c6f60-e6b6-4e62-ae10-cdb0b94661ed admin admin] PATH_INFO: `/v3/projects/3e22a25921ea45eba0f29a87edaba447` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:36:48.643670 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1c6f60-e6b6-4e62-ae10-cdb0b94661ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e22a25921ea45eba0f29a87edaba447)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:36:48.646101 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1c6f60-e6b6-4e62-ae10-cdb0b94661ed admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:36:48.816152 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-fa1c6f60-e6b6-4e62-ae10-cdb0b94661ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e22a25921ea45eba0f29a87edaba447', 'request_id': 'req-fa1c6f60-e6b6-4e62-ae10-cdb0b94661ed'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:36:48.884657 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-fa1c6f60-e6b6-4e62-ae10-cdb0b94661ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e22a25921ea45eba0f29a87edaba447', 'request_id': 'req-fa1c6f60-e6b6-4e62-ae10-cdb0b94661ed'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:36:48.891460 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-fa1c6f60-e6b6-4e62-ae10-cdb0b94661ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e22a25921ea45eba0f29a87edaba447', 'request_id': 'req-fa1c6f60-e6b6-4e62-ae10-cdb0b94661ed'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:36:48.903672 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-fa1c6f60-e6b6-4e62-ae10-cdb0b94661ed admin admin] The token cache is being invalidate because project 3e22a25921ea45eba0f29a87edaba447 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81995) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:36:48.903960 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-fa1c6f60-e6b6-4e62-ae10-cdb0b94661ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:36:48.904598 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-fa1c6f60-e6b6-4e62-ae10-cdb0b94661ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:36:48.906239 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1370/2756] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:36:48 2022] DELETE /identity/v3/projects/3e22a25921ea45eba0f29a87edaba447 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:36:48.911110 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61be2656-a36c-4f7d-bb1e-5d9be5cc583a None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:36:48.911475 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61be2656-a36c-4f7d-bb1e-5d9be5cc583a None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:36:48.911762 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61be2656-a36c-4f7d-bb1e-5d9be5cc583a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:36:48.933579 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-61be2656-a36c-4f7d-bb1e-5d9be5cc583a None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:36:48.934850 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-61be2656-a36c-4f7d-bb1e-5d9be5cc583a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:36:48.996829 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1387/2757] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:36:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:36:49.005617 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd5a6fa-6273-4052-ba53-f640bc041146 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:36:49.006093 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd5a6fa-6273-4052-ba53-f640bc041146 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:36:49.006498 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd5a6fa-6273-4052-ba53-f640bc041146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:36:49.026949 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-4dd5a6fa-6273-4052-ba53-f640bc041146 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:36:49.030499 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-4dd5a6fa-6273-4052-ba53-f640bc041146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:36:49.036087 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1371/2758] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:36:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:36:49.040426 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61be2656-a36c-4f7d-bb1e-5d9be5cc583a None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:36:49.049996 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61be2656-a36c-4f7d-bb1e-5d9be5cc583a None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:36:49.054610 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eea0a5-17ad-4e96-895d-c8affbe44e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:36:49.055206 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eea0a5-17ad-4e96-895d-c8affbe44e29 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:36:49.055485 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eea0a5-17ad-4e96-895d-c8affbe44e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:36:49.055754 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eea0a5-17ad-4e96-895d-c8affbe44e29 admin admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:36:49.056672 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2eea0a5-17ad-4e96-895d-c8affbe44e29 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:36:49.057005 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2eea0a5-17ad-4e96-895d-c8affbe44e29 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:36:49.058598 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2eea0a5-17ad-4e96-895d-c8affbe44e29 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:36:49.060939 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:36:49.071804 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1388/2759] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Aug 25 03:36:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:36:49.075926 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd5a6fa-6273-4052-ba53-f640bc041146 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:36:49.085959 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd5a6fa-6273-4052-ba53-f640bc041146 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:36:49.090776 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e76fba-523d-4898-83bd-3cc8dedec501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:36:49.091382 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e76fba-523d-4898-83bd-3cc8dedec501 admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:36:49.091660 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e76fba-523d-4898-83bd-3cc8dedec501 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:36:49.091930 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e76fba-523d-4898-83bd-3cc8dedec501 admin admin] PATH_INFO: `/v3/projects` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:36:49.092554 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e76fba-523d-4898-83bd-3cc8dedec501 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:36:49.094495 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e76fba-523d-4898-83bd-3cc8dedec501 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:36:49.177965 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1372/2760] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:36:49 2022] POST /identity/v3/projects => generated 371 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:36:49.182724 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eea0a5-17ad-4e96-895d-c8affbe44e29 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:36:49.192578 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eea0a5-17ad-4e96-895d-c8affbe44e29 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:36:49.251564 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcdefb3-443c-4fc8-98b5-cea6074d8fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:36:49.252186 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcdefb3-443c-4fc8-98b5-cea6074d8fc3 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:36:49.252471 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcdefb3-443c-4fc8-98b5-cea6074d8fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:36:49.252749 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcdefb3-443c-4fc8-98b5-cea6074d8fc3 admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:36:49.253383 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcdefb3-443c-4fc8-98b5-cea6074d8fc3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:36:49.255385 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcdefb3-443c-4fc8-98b5-cea6074d8fc3 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:36:49.335377 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1389/2761] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:36:49 2022] POST /identity/v3/users => generated 286 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:36:49.336589 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e76fba-523d-4898-83bd-3cc8dedec501 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:36:49.347417 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e76fba-523d-4898-83bd-3cc8dedec501 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:36:49.353429 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d24e35b-e294-4b8f-8c2d-ed8ffc349222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:36:49.354679 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d24e35b-e294-4b8f-8c2d-ed8ffc349222 admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:36:49.354679 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d24e35b-e294-4b8f-8c2d-ed8ffc349222 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:36:49.354679 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d24e35b-e294-4b8f-8c2d-ed8ffc349222 admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:36:49.355254 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d24e35b-e294-4b8f-8c2d-ed8ffc349222 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:36:49.356843 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d24e35b-e294-4b8f-8c2d-ed8ffc349222 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:36:49.367272 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1373/2762] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:36:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:36:49.371315 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcdefb3-443c-4fc8-98b5-cea6074d8fc3 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:36:49.381481 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcdefb3-443c-4fc8-98b5-cea6074d8fc3 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:36:49.386150 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670c0080-5a81-458f-bdec-be028b8fb452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:36:49.386786 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670c0080-5a81-458f-bdec-be028b8fb452 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:36:49.387065 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670c0080-5a81-458f-bdec-be028b8fb452 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:36:49.387337 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670c0080-5a81-458f-bdec-be028b8fb452 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:36:49.388127 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670c0080-5a81-458f-bdec-be028b8fb452 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:36:49.389722 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670c0080-5a81-458f-bdec-be028b8fb452 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:36:49.401154 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1390/2763] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:36:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:36:49.405463 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d24e35b-e294-4b8f-8c2d-ed8ffc349222 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:36:49.416294 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d24e35b-e294-4b8f-8c2d-ed8ffc349222 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:36:49.421147 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736ccc63-dee2-451e-8006-86932f8ceaf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:36:49.429159 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736ccc63-dee2-451e-8006-86932f8ceaf8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:36:49.429159 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736ccc63-dee2-451e-8006-86932f8ceaf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:36:49.429159 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736ccc63-dee2-451e-8006-86932f8ceaf8 admin admin] PATH_INFO: `/v3/projects/8aeeca6ebe2e45f088d2cf441b81e305/users/31586101033d47a59bc85e25fbd5b352/roles/3255f77192524a16bc405173106871fc` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:36:49.439045 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736ccc63-dee2-451e-8006-86932f8ceaf8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8aeeca6ebe2e45f088d2cf441b81e305, user_id=31586101033d47a59bc85e25fbd5b352, role_id=3255f77192524a16bc405173106871fc)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:36:49.441698 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736ccc63-dee2-451e-8006-86932f8ceaf8 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:36:49.508128 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1374/2764] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:36:49 2022] PUT /identity/v3/projects/8aeeca6ebe2e45f088d2cf441b81e305/users/31586101033d47a59bc85e25fbd5b352/roles/3255f77192524a16bc405173106871fc => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:36:49.518808 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2234cbae-d48a-487d-8e46-9ae585b1095e None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:36:49.519240 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2234cbae-d48a-487d-8e46-9ae585b1095e None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:36:49.519596 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2234cbae-d48a-487d-8e46-9ae585b1095e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:36:49.548378 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-2234cbae-d48a-487d-8e46-9ae585b1095e None None] MFA Rules not processed for user `31586101033d47a59bc85e25fbd5b352`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:36:49.549899 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-2234cbae-d48a-487d-8e46-9ae585b1095e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:36:49.616834 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1391/2765] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:36:49 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:36:49.621149 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736ccc63-dee2-451e-8006-86932f8ceaf8 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:36:49.635729 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:36:49.686871 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736ccc63-dee2-451e-8006-86932f8ceaf8 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:36:49.697566 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad661bb-22fa-4e40-a586-74dde76455a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:36:49.701451 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad661bb-22fa-4e40-a586-74dde76455a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:36:49.701451 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad661bb-22fa-4e40-a586-74dde76455a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:36:49.701451 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad661bb-22fa-4e40-a586-74dde76455a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:36:49.701451 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad661bb-22fa-4e40-a586-74dde76455a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:36:49.701451 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad661bb-22fa-4e40-a586-74dde76455a1 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:36:49.816684 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1375/2766] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:36:49 2022] POST /identity/v3/projects => generated 373 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:36:49.819569 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2234cbae-d48a-487d-8e46-9ae585b1095e None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:36:49.833694 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad661bb-22fa-4e40-a586-74dde76455a1 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:36:49.835014 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-bad661bb-22fa-4e40-a586-74dde76455a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:36:49.839570 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2234cbae-d48a-487d-8e46-9ae585b1095e None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:36:49.906994 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ed033f-8b2e-4cf2-89b2-031521794066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:36:49.907692 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ed033f-8b2e-4cf2-89b2-031521794066 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:36:49.907979 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ed033f-8b2e-4cf2-89b2-031521794066 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:36:49.908253 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ed033f-8b2e-4cf2-89b2-031521794066 admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:36:49.908901 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ed033f-8b2e-4cf2-89b2-031521794066 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:36:49.910955 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ed033f-8b2e-4cf2-89b2-031521794066 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:36:49.921633 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:36:49.930090 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad661bb-22fa-4e40-a586-74dde76455a1 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:36:49.936079 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1d7648-61b6-4439-a953-31674a559ec7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cd171bf4df94f8fab397e42835feeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:36:49.936635 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1d7648-61b6-4439-a953-31674a559ec7 service nova] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:36:49.936916 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1d7648-61b6-4439-a953-31674a559ec7 service nova] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:36:49.937188 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1d7648-61b6-4439-a953-31674a559ec7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:36:49.938964 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-8d1d7648-61b6-4439-a953-31674a559ec7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:36:49.952735 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1d7648-61b6-4439-a953-31674a559ec7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:36:49.954410 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1d7648-61b6-4439-a953-31674a559ec7 service nova] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:36:49.967208 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1376/2767] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:36:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:36:50.018320 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1392/2768] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:36:49 2022] POST /identity/v3/users => generated 293 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:36:50.019725 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1d7648-61b6-4439-a953-31674a559ec7 service nova] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:36:50.030140 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1d7648-61b6-4439-a953-31674a559ec7 service nova] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:36:50.034936 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b278681-9d45-4d50-a7aa-e7909690cf73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:36:50.035543 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b278681-9d45-4d50-a7aa-e7909690cf73 admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:36:50.035849 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b278681-9d45-4d50-a7aa-e7909690cf73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:36:50.036125 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b278681-9d45-4d50-a7aa-e7909690cf73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:36:50.036771 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b278681-9d45-4d50-a7aa-e7909690cf73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:36:50.038425 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b278681-9d45-4d50-a7aa-e7909690cf73 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:36:50.055921 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1377/2769] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:36:50 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:36:50.057732 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ed033f-8b2e-4cf2-89b2-031521794066 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:36:50.068874 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ed033f-8b2e-4cf2-89b2-031521794066 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:36:50.076952 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8166616e-3fff-4412-a5ef-f38bd39bed6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:36:50.077715 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8166616e-3fff-4412-a5ef-f38bd39bed6e admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:36:50.078551 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8166616e-3fff-4412-a5ef-f38bd39bed6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:36:50.078923 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8166616e-3fff-4412-a5ef-f38bd39bed6e admin admin] PATH_INFO: `/v3/projects/03950b3c0f3d47bea2ca12824e53afd6/users/ff861812a2154ae79b9d5a16bd462203/roles/62f19ea272b447ccaab1576cd9bf4fd9` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:36:50.095624 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8166616e-3fff-4412-a5ef-f38bd39bed6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=03950b3c0f3d47bea2ca12824e53afd6, user_id=ff861812a2154ae79b9d5a16bd462203, role_id=62f19ea272b447ccaab1576cd9bf4fd9)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:36:50.098773 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8166616e-3fff-4412-a5ef-f38bd39bed6e admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:36:50.178180 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1393/2770] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:36:50 2022] PUT /identity/v3/projects/03950b3c0f3d47bea2ca12824e53afd6/users/ff861812a2154ae79b9d5a16bd462203/roles/62f19ea272b447ccaab1576cd9bf4fd9 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:36:50.182555 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b278681-9d45-4d50-a7aa-e7909690cf73 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:36:50.195086 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b278681-9d45-4d50-a7aa-e7909690cf73 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:36:50.261970 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb0a9b7-121e-4d23-8e56-8e50f8d50f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:36:50.262632 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb0a9b7-121e-4d23-8e56-8e50f8d50f99 admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:36:50.262911 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb0a9b7-121e-4d23-8e56-8e50f8d50f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:36:50.263185 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb0a9b7-121e-4d23-8e56-8e50f8d50f99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:36:50.263836 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb0a9b7-121e-4d23-8e56-8e50f8d50f99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:36:50.265438 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb0a9b7-121e-4d23-8e56-8e50f8d50f99 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:36:50.278203 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1378/2771] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:36:50 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 25 03:36:50.282465 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8166616e-3fff-4412-a5ef-f38bd39bed6e admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:36:50.295322 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8166616e-3fff-4412-a5ef-f38bd39bed6e admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:36:50.300639 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3611704e-c0c8-4203-b7d3-c4223ab15532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:36:50.301682 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3611704e-c0c8-4203-b7d3-c4223ab15532 admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:36:50.301965 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3611704e-c0c8-4203-b7d3-c4223ab15532 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:36:50.302241 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3611704e-c0c8-4203-b7d3-c4223ab15532 admin admin] PATH_INFO: `/v3/projects/03950b3c0f3d47bea2ca12824e53afd6/users/ff861812a2154ae79b9d5a16bd462203/roles/62f19ea272b447ccaab1576cd9bf4fd9` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:36:50.304498 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3611704e-c0c8-4203-b7d3-c4223ab15532 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03950b3c0f3d47bea2ca12824e53afd6, user_id=ff861812a2154ae79b9d5a16bd462203, role_id=62f19ea272b447ccaab1576cd9bf4fd9)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:36:50.308305 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3611704e-c0c8-4203-b7d3-c4223ab15532 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:36:50.319962 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1394/2772] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:36:50 2022] PUT /identity/v3/projects/03950b3c0f3d47bea2ca12824e53afd6/users/ff861812a2154ae79b9d5a16bd462203/roles/62f19ea272b447ccaab1576cd9bf4fd9 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:36:50.332355 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abad0264-785c-42e6-92a2-fbfc4edd9d35 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:36:50.332690 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abad0264-785c-42e6-92a2-fbfc4edd9d35 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:36:50.333575 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abad0264-785c-42e6-92a2-fbfc4edd9d35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:36:50.356454 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:36:50.366926 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-abad0264-785c-42e6-92a2-fbfc4edd9d35 None None] MFA Rules not processed for user `ff861812a2154ae79b9d5a16bd462203`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:36:50.369014 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-abad0264-785c-42e6-92a2-fbfc4edd9d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:36:50.443783 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1379/2773] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:36:50 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:36:50.457277 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3611704e-c0c8-4203-b7d3-c4223ab15532 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:36:50.459764 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-3611704e-c0c8-4203-b7d3-c4223ab15532 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:36:50.525860 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3611704e-c0c8-4203-b7d3-c4223ab15532 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:36:50.531348 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64614e46-345e-4622-a1b2-3aebb23d9dc6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60193ba9e1c14df3bc9eebcc6f28f2d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:36:50.532000 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64614e46-345e-4622-a1b2-3aebb23d9dc6 service cinder] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:36:50.532284 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64614e46-345e-4622-a1b2-3aebb23d9dc6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:36:50.532562 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64614e46-345e-4622-a1b2-3aebb23d9dc6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:36:50.545619 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64614e46-345e-4622-a1b2-3aebb23d9dc6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:36:50.547177 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64614e46-345e-4622-a1b2-3aebb23d9dc6 service cinder] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:36:50.561053 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1395/2774] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:36:50 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:36:51.495758 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abad0264-785c-42e6-92a2-fbfc4edd9d35 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:36:51.556172 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abad0264-785c-42e6-92a2-fbfc4edd9d35 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:36:51.560884 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac90a00e-6bc6-4b3b-96e9-aaae51009cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:36:51.561505 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac90a00e-6bc6-4b3b-96e9-aaae51009cf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:36:51.561782 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac90a00e-6bc6-4b3b-96e9-aaae51009cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:36:51.562055 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac90a00e-6bc6-4b3b-96e9-aaae51009cf8 admin admin] PATH_INFO: `/v3/users/31586101033d47a59bc85e25fbd5b352` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:36:51.563184 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac90a00e-6bc6-4b3b-96e9-aaae51009cf8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=31586101033d47a59bc85e25fbd5b352)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:36:51.565094 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac90a00e-6bc6-4b3b-96e9-aaae51009cf8 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:36:51.738746 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-ac90a00e-6bc6-4b3b-96e9-aaae51009cf8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31586101033d47a59bc85e25fbd5b352'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:36:51.815929 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-ac90a00e-6bc6-4b3b-96e9-aaae51009cf8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31586101033d47a59bc85e25fbd5b352'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:36:51.816499 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-ac90a00e-6bc6-4b3b-96e9-aaae51009cf8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31586101033d47a59bc85e25fbd5b352'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:36:51.817654 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-ac90a00e-6bc6-4b3b-96e9-aaae51009cf8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31586101033d47a59bc85e25fbd5b352'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:36:51.830120 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-ac90a00e-6bc6-4b3b-96e9-aaae51009cf8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31586101033d47a59bc85e25fbd5b352'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:36:51.842993 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1380/2775] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:36:51 2022] DELETE /identity/v3/users/31586101033d47a59bc85e25fbd5b352 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:36:51.847570 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9222f11-32f0-4923-b7ee-1d530dd93940 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:36:51.848012 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9222f11-32f0-4923-b7ee-1d530dd93940 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:36:51.848119 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9222f11-32f0-4923-b7ee-1d530dd93940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:36:51.866961 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-d9222f11-32f0-4923-b7ee-1d530dd93940 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:36:51.868152 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-d9222f11-32f0-4923-b7ee-1d530dd93940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:36:51.924850 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1396/2776] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:36:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:36:51.935845 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac90a00e-6bc6-4b3b-96e9-aaae51009cf8 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:36:51.939576 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-ac90a00e-6bc6-4b3b-96e9-aaae51009cf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:36:51.954466 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:36:51.998048 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac90a00e-6bc6-4b3b-96e9-aaae51009cf8 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:36:52.001604 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc88d05-3263-4bb0-9427-2ebdaca7e2cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:36:52.002204 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc88d05-3263-4bb0-9427-2ebdaca7e2cf service neutron] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:36:52.002511 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc88d05-3263-4bb0-9427-2ebdaca7e2cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:36:52.002781 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc88d05-3263-4bb0-9427-2ebdaca7e2cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:36:52.011843 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc88d05-3263-4bb0-9427-2ebdaca7e2cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:36:52.013262 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc88d05-3263-4bb0-9427-2ebdaca7e2cf service neutron] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:36:52.023249 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1381/2777] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:36:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:36:52.429464 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9222f11-32f0-4923-b7ee-1d530dd93940 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:36:52.432530 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-d9222f11-32f0-4923-b7ee-1d530dd93940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:36:52.443677 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9222f11-32f0-4923-b7ee-1d530dd93940 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:36:52.448272 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6f3256-4d79-46b0-83fd-32c82886519b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:36:52.448895 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6f3256-4d79-46b0-83fd-32c82886519b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:36:52.449173 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6f3256-4d79-46b0-83fd-32c82886519b admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:36:52.449441 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6f3256-4d79-46b0-83fd-32c82886519b admin admin] PATH_INFO: `/v3/projects/8aeeca6ebe2e45f088d2cf441b81e305` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:36:52.450630 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6f3256-4d79-46b0-83fd-32c82886519b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8aeeca6ebe2e45f088d2cf441b81e305)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:36:52.452512 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6f3256-4d79-46b0-83fd-32c82886519b admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:36:52.561271 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-9e6f3256-4d79-46b0-83fd-32c82886519b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8aeeca6ebe2e45f088d2cf441b81e305', 'request_id': 'req-9e6f3256-4d79-46b0-83fd-32c82886519b'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:36:52.563848 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:INSERT=5,keystone:SELECT=91,keystone:DELETE=4 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:36:52.639014 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-9e6f3256-4d79-46b0-83fd-32c82886519b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8aeeca6ebe2e45f088d2cf441b81e305', 'request_id': 'req-9e6f3256-4d79-46b0-83fd-32c82886519b'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:36:52.645115 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-9e6f3256-4d79-46b0-83fd-32c82886519b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8aeeca6ebe2e45f088d2cf441b81e305', 'request_id': 'req-9e6f3256-4d79-46b0-83fd-32c82886519b'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:36:52.656102 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-9e6f3256-4d79-46b0-83fd-32c82886519b admin admin] The token cache is being invalidate because project 8aeeca6ebe2e45f088d2cf441b81e305 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:36:52.660463 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-9e6f3256-4d79-46b0-83fd-32c82886519b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:36:52.660463 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-9e6f3256-4d79-46b0-83fd-32c82886519b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:36:52.660463 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1397/2778] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:36:52 2022] DELETE /identity/v3/projects/8aeeca6ebe2e45f088d2cf441b81e305 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:36:52.663254 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc88d05-3263-4bb0-9427-2ebdaca7e2cf service neutron] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:36:52.673102 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc88d05-3263-4bb0-9427-2ebdaca7e2cf service neutron] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:36:52.728780 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acefbb48-f2e5-4a41-bebd-4bfd5057e5c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:36:52.729417 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acefbb48-f2e5-4a41-bebd-4bfd5057e5c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:36:52.729701 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acefbb48-f2e5-4a41-bebd-4bfd5057e5c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:36:52.729974 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acefbb48-f2e5-4a41-bebd-4bfd5057e5c5 admin admin] PATH_INFO: `/v3/users/ff861812a2154ae79b9d5a16bd462203` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:36:52.731215 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acefbb48-f2e5-4a41-bebd-4bfd5057e5c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff861812a2154ae79b9d5a16bd462203)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:36:52.733130 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acefbb48-f2e5-4a41-bebd-4bfd5057e5c5 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:36:52.880366 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-acefbb48-f2e5-4a41-bebd-4bfd5057e5c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff861812a2154ae79b9d5a16bd462203'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:36:52.924481 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-acefbb48-f2e5-4a41-bebd-4bfd5057e5c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff861812a2154ae79b9d5a16bd462203'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:36:52.925184 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-acefbb48-f2e5-4a41-bebd-4bfd5057e5c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff861812a2154ae79b9d5a16bd462203'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:36:52.925795 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-acefbb48-f2e5-4a41-bebd-4bfd5057e5c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff861812a2154ae79b9d5a16bd462203'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:36:52.938723 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-acefbb48-f2e5-4a41-bebd-4bfd5057e5c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff861812a2154ae79b9d5a16bd462203'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:36:52.950682 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1382/2779] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:36:52 2022] DELETE /identity/v3/users/ff861812a2154ae79b9d5a16bd462203 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:36:53.331465 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6f3256-4d79-46b0-83fd-32c82886519b admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:36:53.332949 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-9e6f3256-4d79-46b0-83fd-32c82886519b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:36:53.346612 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6f3256-4d79-46b0-83fd-32c82886519b admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:36:53.404032 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23abfbb-ae6d-4bf6-9a8e-a96679ee6755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:36:53.404671 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23abfbb-ae6d-4bf6-9a8e-a96679ee6755 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:36:53.404952 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23abfbb-ae6d-4bf6-9a8e-a96679ee6755 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:36:53.405229 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23abfbb-ae6d-4bf6-9a8e-a96679ee6755 admin admin] PATH_INFO: `/v3/projects/03950b3c0f3d47bea2ca12824e53afd6` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:36:53.406369 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23abfbb-ae6d-4bf6-9a8e-a96679ee6755 admin admin] RBAC: Authorizing `identity:delete_project(project_id=03950b3c0f3d47bea2ca12824e53afd6)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:36:53.408259 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23abfbb-ae6d-4bf6-9a8e-a96679ee6755 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:36:53.513272 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-a23abfbb-ae6d-4bf6-9a8e-a96679ee6755 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03950b3c0f3d47bea2ca12824e53afd6', 'request_id': 'req-a23abfbb-ae6d-4bf6-9a8e-a96679ee6755'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:36:53.593067 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-a23abfbb-ae6d-4bf6-9a8e-a96679ee6755 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03950b3c0f3d47bea2ca12824e53afd6', 'request_id': 'req-a23abfbb-ae6d-4bf6-9a8e-a96679ee6755'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:36:53.599191 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-a23abfbb-ae6d-4bf6-9a8e-a96679ee6755 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03950b3c0f3d47bea2ca12824e53afd6', 'request_id': 'req-a23abfbb-ae6d-4bf6-9a8e-a96679ee6755'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:36:53.612062 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-a23abfbb-ae6d-4bf6-9a8e-a96679ee6755 admin admin] The token cache is being invalidate because project 03950b3c0f3d47bea2ca12824e53afd6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:36:53.612349 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-a23abfbb-ae6d-4bf6-9a8e-a96679ee6755 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:36:53.612999 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-a23abfbb-ae6d-4bf6-9a8e-a96679ee6755 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:36:53.614591 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1398/2780] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:36:53 2022] DELETE /identity/v3/projects/03950b3c0f3d47bea2ca12824e53afd6 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:36:53.619545 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7e5c81-8e1e-4cf0-b851-7a98ed8ac66c None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:36:53.619836 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7e5c81-8e1e-4cf0-b851-7a98ed8ac66c None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:36:53.620117 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7e5c81-8e1e-4cf0-b851-7a98ed8ac66c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:36:53.640686 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-8c7e5c81-8e1e-4cf0-b851-7a98ed8ac66c None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:36:53.642479 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-8c7e5c81-8e1e-4cf0-b851-7a98ed8ac66c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:36:53.672219 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:36:53.702269 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1383/2781] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:36:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:36:53.709994 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b35600-ed97-4017-918a-8d5809d3984a None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:36:53.710294 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b35600-ed97-4017-918a-8d5809d3984a None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:36:53.710589 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b35600-ed97-4017-918a-8d5809d3984a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:36:53.730098 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-c5b35600-ed97-4017-918a-8d5809d3984a None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:36:53.731313 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-c5b35600-ed97-4017-918a-8d5809d3984a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:36:53.740594 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1399/2782] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:36:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:36:53.744545 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7e5c81-8e1e-4cf0-b851-7a98ed8ac66c None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:36:53.754312 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7e5c81-8e1e-4cf0-b851-7a98ed8ac66c None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:36:53.761771 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb9c8eb-06e8-4301-8979-8a9e1a6a82e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:36:53.761771 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb9c8eb-06e8-4301-8979-8a9e1a6a82e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:36:53.761771 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb9c8eb-06e8-4301-8979-8a9e1a6a82e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:36:53.761771 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb9c8eb-06e8-4301-8979-8a9e1a6a82e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:36:53.762193 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb9c8eb-06e8-4301-8979-8a9e1a6a82e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:36:53.762193 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb9c8eb-06e8-4301-8979-8a9e1a6a82e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:36:53.763630 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb9c8eb-06e8-4301-8979-8a9e1a6a82e5 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:36:53.775208 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1384/2783] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Aug 25 03:36:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:36:53.779454 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b35600-ed97-4017-918a-8d5809d3984a None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:36:53.788618 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b35600-ed97-4017-918a-8d5809d3984a None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:36:53.793155 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647eb4d0-852d-4e36-9721-d7d230382fca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:36:53.793754 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647eb4d0-852d-4e36-9721-d7d230382fca admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:36:53.794032 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647eb4d0-852d-4e36-9721-d7d230382fca admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:36:53.794348 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647eb4d0-852d-4e36-9721-d7d230382fca admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:36:53.794980 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647eb4d0-852d-4e36-9721-d7d230382fca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:36:53.796885 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647eb4d0-852d-4e36-9721-d7d230382fca admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:36:53.920463 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1400/2784] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:36:53 2022] POST /identity/v3/projects => generated 379 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:36:53.924634 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb9c8eb-06e8-4301-8979-8a9e1a6a82e5 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:36:53.935476 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb9c8eb-06e8-4301-8979-8a9e1a6a82e5 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:36:53.990964 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989a0bc0-fab1-4118-bc20-b76bbc435d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:36:53.991566 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989a0bc0-fab1-4118-bc20-b76bbc435d56 admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:36:53.991881 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989a0bc0-fab1-4118-bc20-b76bbc435d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:36:53.992135 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989a0bc0-fab1-4118-bc20-b76bbc435d56 admin admin] PATH_INFO: `/v3/users` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:36:53.992753 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989a0bc0-fab1-4118-bc20-b76bbc435d56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:36:53.994698 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989a0bc0-fab1-4118-bc20-b76bbc435d56 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:36:54.101922 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1385/2785] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:36:53 2022] POST /identity/v3/users => generated 290 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:36:54.105965 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647eb4d0-852d-4e36-9721-d7d230382fca admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:36:54.116943 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647eb4d0-852d-4e36-9721-d7d230382fca admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:36:54.122257 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e0ef25-d07e-416c-bb85-8705a7390fdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:36:54.122888 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e0ef25-d07e-416c-bb85-8705a7390fdc admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:36:54.123210 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e0ef25-d07e-416c-bb85-8705a7390fdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:36:54.123485 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e0ef25-d07e-416c-bb85-8705a7390fdc admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:36:54.124161 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e0ef25-d07e-416c-bb85-8705a7390fdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:36:54.125734 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e0ef25-d07e-416c-bb85-8705a7390fdc admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:36:54.135718 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1401/2786] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:36:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:36:54.139595 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989a0bc0-fab1-4118-bc20-b76bbc435d56 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:36:54.149771 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989a0bc0-fab1-4118-bc20-b76bbc435d56 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:36:54.154412 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ceebba-5613-4572-90ee-0a4e1f014420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:36:54.155030 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ceebba-5613-4572-90ee-0a4e1f014420 admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:36:54.155296 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ceebba-5613-4572-90ee-0a4e1f014420 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:36:54.155572 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ceebba-5613-4572-90ee-0a4e1f014420 admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:36:54.156218 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ceebba-5613-4572-90ee-0a4e1f014420 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:36:54.157816 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ceebba-5613-4572-90ee-0a4e1f014420 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:36:54.168785 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1386/2787] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:36:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:36:54.172751 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e0ef25-d07e-416c-bb85-8705a7390fdc admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:36:54.183811 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e0ef25-d07e-416c-bb85-8705a7390fdc admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:36:54.189638 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aef2e0-517e-4b47-b7c0-2f889d9cd34f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:36:54.190417 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aef2e0-517e-4b47-b7c0-2f889d9cd34f admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:36:54.190677 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aef2e0-517e-4b47-b7c0-2f889d9cd34f admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:36:54.190954 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aef2e0-517e-4b47-b7c0-2f889d9cd34f admin admin] PATH_INFO: `/v3/projects/c171b9a284c24ff19b13626187c088b2/users/a12495f4993244b6983a1ea4d385376c/roles/3255f77192524a16bc405173106871fc` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:36:54.205785 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2aef2e0-517e-4b47-b7c0-2f889d9cd34f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c171b9a284c24ff19b13626187c088b2, user_id=a12495f4993244b6983a1ea4d385376c, role_id=3255f77192524a16bc405173106871fc)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:36:54.208482 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2aef2e0-517e-4b47-b7c0-2f889d9cd34f admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:36:54.302061 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1402/2788] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:36:54 2022] PUT /identity/v3/projects/c171b9a284c24ff19b13626187c088b2/users/a12495f4993244b6983a1ea4d385376c/roles/3255f77192524a16bc405173106871fc => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:36:54.314069 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a81a444-0067-4a1a-9a19-eddb9f61c372 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:36:54.314385 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a81a444-0067-4a1a-9a19-eddb9f61c372 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:36:54.314663 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a81a444-0067-4a1a-9a19-eddb9f61c372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:36:54.341888 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-7a81a444-0067-4a1a-9a19-eddb9f61c372 None None] MFA Rules not processed for user `a12495f4993244b6983a1ea4d385376c`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:36:54.343175 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-7a81a444-0067-4a1a-9a19-eddb9f61c372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:36:54.405495 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1387/2789] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:36:54 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:36:54.410006 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aef2e0-517e-4b47-b7c0-2f889d9cd34f admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:36:54.419315 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:DELETE=10,keystone:SELECT=87,keystone:INSERT=3 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:36:54.472101 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aef2e0-517e-4b47-b7c0-2f889d9cd34f admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:36:54.477498 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f42b7ea-d4f3-4f59-a736-e2bae6867801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:36:54.478099 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f42b7ea-d4f3-4f59-a736-e2bae6867801 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:36:54.478409 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f42b7ea-d4f3-4f59-a736-e2bae6867801 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:36:54.478684 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f42b7ea-d4f3-4f59-a736-e2bae6867801 admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:36:54.479311 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f42b7ea-d4f3-4f59-a736-e2bae6867801 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:36:54.481192 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f42b7ea-d4f3-4f59-a736-e2bae6867801 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:36:54.611813 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1403/2790] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:36:54 2022] POST /identity/v3/projects => generated 381 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:36:54.615976 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a81a444-0067-4a1a-9a19-eddb9f61c372 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:36:54.626671 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a81a444-0067-4a1a-9a19-eddb9f61c372 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:36:54.655150 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:36:54.681942 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287df58f-488c-409c-838b-0dc3000789df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:36:54.682572 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287df58f-488c-409c-838b-0dc3000789df admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:36:54.682832 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287df58f-488c-409c-838b-0dc3000789df admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:36:54.683145 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287df58f-488c-409c-838b-0dc3000789df admin admin] PATH_INFO: `/v3/users` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:36:54.683762 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287df58f-488c-409c-838b-0dc3000789df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:36:54.685699 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287df58f-488c-409c-838b-0dc3000789df admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:36:54.806182 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1388/2791] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:36:54 2022] POST /identity/v3/users => generated 297 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:36:54.810915 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f42b7ea-d4f3-4f59-a736-e2bae6867801 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:36:54.823735 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f42b7ea-d4f3-4f59-a736-e2bae6867801 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:36:54.828432 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f346c9f-8a94-4118-a990-7d8de15256bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:36:54.829053 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f346c9f-8a94-4118-a990-7d8de15256bd admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:36:54.829333 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f346c9f-8a94-4118-a990-7d8de15256bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:36:54.829614 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f346c9f-8a94-4118-a990-7d8de15256bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:36:54.830250 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f346c9f-8a94-4118-a990-7d8de15256bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:36:54.831891 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f346c9f-8a94-4118-a990-7d8de15256bd admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:36:54.843295 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1404/2792] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:36:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 25 03:36:54.847423 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287df58f-488c-409c-838b-0dc3000789df admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:36:54.857314 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287df58f-488c-409c-838b-0dc3000789df admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:36:54.861838 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825b3431-0b6a-4d82-9d86-d804f6454f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:36:54.862517 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825b3431-0b6a-4d82-9d86-d804f6454f0e admin admin] REQUEST_METHOD: `PUT` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:36:54.862796 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825b3431-0b6a-4d82-9d86-d804f6454f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:36:54.863080 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825b3431-0b6a-4d82-9d86-d804f6454f0e admin admin] PATH_INFO: `/v3/projects/1219a34b406149b8931fb9f76f1ee82c/users/d85db2a953f74e578269be1476633b72/roles/62f19ea272b447ccaab1576cd9bf4fd9` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:36:54.877908 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825b3431-0b6a-4d82-9d86-d804f6454f0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1219a34b406149b8931fb9f76f1ee82c, user_id=d85db2a953f74e578269be1476633b72, role_id=62f19ea272b447ccaab1576cd9bf4fd9)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:36:54.881295 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825b3431-0b6a-4d82-9d86-d804f6454f0e admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:36:54.947488 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1389/2793] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:36:54 2022] PUT /identity/v3/projects/1219a34b406149b8931fb9f76f1ee82c/users/d85db2a953f74e578269be1476633b72/roles/62f19ea272b447ccaab1576cd9bf4fd9 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:36:54.951411 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f346c9f-8a94-4118-a990-7d8de15256bd admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:36:54.961879 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f346c9f-8a94-4118-a990-7d8de15256bd admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:36:55.015943 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6094b230-8429-45e1-b21c-a1eb68d901d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:36:55.016578 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6094b230-8429-45e1-b21c-a1eb68d901d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:36:55.016920 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6094b230-8429-45e1-b21c-a1eb68d901d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:36:55.017202 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6094b230-8429-45e1-b21c-a1eb68d901d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:36:55.017841 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6094b230-8429-45e1-b21c-a1eb68d901d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:36:55.019460 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6094b230-8429-45e1-b21c-a1eb68d901d8 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:36:55.033536 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1405/2794] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:36:54 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:36:55.034914 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825b3431-0b6a-4d82-9d86-d804f6454f0e admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:36:55.045264 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825b3431-0b6a-4d82-9d86-d804f6454f0e admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:36:55.049876 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa556f6-a565-44ed-94a7-73d899ad3fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:36:55.050554 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa556f6-a565-44ed-94a7-73d899ad3fed admin admin] REQUEST_METHOD: `PUT` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:36:55.050837 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa556f6-a565-44ed-94a7-73d899ad3fed admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:36:55.051115 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa556f6-a565-44ed-94a7-73d899ad3fed admin admin] PATH_INFO: `/v3/projects/1219a34b406149b8931fb9f76f1ee82c/users/d85db2a953f74e578269be1476633b72/roles/62f19ea272b447ccaab1576cd9bf4fd9` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:36:55.053040 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa556f6-a565-44ed-94a7-73d899ad3fed admin admin] RBAC: Authorizing `identity:create_grant(project_id=1219a34b406149b8931fb9f76f1ee82c, user_id=d85db2a953f74e578269be1476633b72, role_id=62f19ea272b447ccaab1576cd9bf4fd9)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:36:55.055739 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa556f6-a565-44ed-94a7-73d899ad3fed admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:36:55.065968 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1390/2795] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:36:55 2022] PUT /identity/v3/projects/1219a34b406149b8931fb9f76f1ee82c/users/d85db2a953f74e578269be1476633b72/roles/62f19ea272b447ccaab1576cd9bf4fd9 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:36:55.127311 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2c97f1-df1b-4f65-bcb6-99ea5099a56b None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:36:55.127599 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2c97f1-df1b-4f65-bcb6-99ea5099a56b None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:36:55.127906 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2c97f1-df1b-4f65-bcb6-99ea5099a56b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:36:55.156413 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-0f2c97f1-df1b-4f65-bcb6-99ea5099a56b None None] MFA Rules not processed for user `d85db2a953f74e578269be1476633b72`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:36:55.157626 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-0f2c97f1-df1b-4f65-bcb6-99ea5099a56b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:36:55.227011 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1406/2796] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:36:55 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:36:55.240142 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa556f6-a565-44ed-94a7-73d899ad3fed admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:36:55.241999 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-daa556f6-a565-44ed-94a7-73d899ad3fed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:36:55.300187 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa556f6-a565-44ed-94a7-73d899ad3fed admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:36:55.303869 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8a6505-5cee-4ba7-a160-099c45f3cfe2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60193ba9e1c14df3bc9eebcc6f28f2d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:36:55.304469 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8a6505-5cee-4ba7-a160-099c45f3cfe2 service cinder] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:36:55.304747 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8a6505-5cee-4ba7-a160-099c45f3cfe2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:36:55.305017 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8a6505-5cee-4ba7-a160-099c45f3cfe2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:36:55.313966 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8a6505-5cee-4ba7-a160-099c45f3cfe2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:36:55.315419 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8a6505-5cee-4ba7-a160-099c45f3cfe2 service cinder] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:36:55.324641 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1391/2797] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:36:55 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:36:56.270419 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2c97f1-df1b-4f65-bcb6-99ea5099a56b None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:36:56.274929 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:36:56.280727 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2c97f1-df1b-4f65-bcb6-99ea5099a56b None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:36:56.285615 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffbf9c4-9893-49cc-b599-5b135aba5e16 tempest-VolumeQuotaClassesTest-1619203118 tempest-VolumeQuotaClassesTest-1619203118-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85db2a953f74e578269be1476633b72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1219a34b406149b8931fb9f76f1ee82c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:36:56.286218 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffbf9c4-9893-49cc-b599-5b135aba5e16 tempest-VolumeQuotaClassesTest-1619203118 tempest-VolumeQuotaClassesTest-1619203118-project-admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:36:56.288022 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffbf9c4-9893-49cc-b599-5b135aba5e16 tempest-VolumeQuotaClassesTest-1619203118 tempest-VolumeQuotaClassesTest-1619203118-project-admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:36:56.288022 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffbf9c4-9893-49cc-b599-5b135aba5e16 tempest-VolumeQuotaClassesTest-1619203118 tempest-VolumeQuotaClassesTest-1619203118-project-admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:36:56.288022 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffbf9c4-9893-49cc-b599-5b135aba5e16 tempest-VolumeQuotaClassesTest-1619203118 tempest-VolumeQuotaClassesTest-1619203118-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:36:56.288427 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffbf9c4-9893-49cc-b599-5b135aba5e16 tempest-VolumeQuotaClassesTest-1619203118 tempest-VolumeQuotaClassesTest-1619203118-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:36:56.289851 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffbf9c4-9893-49cc-b599-5b135aba5e16 tempest-VolumeQuotaClassesTest-1619203118 tempest-VolumeQuotaClassesTest-1619203118-project-admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:36:56.301209 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1407/2798] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Aug 25 03:36:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:36:56.305931 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8a6505-5cee-4ba7-a160-099c45f3cfe2 service cinder] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:36:56.315896 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8a6505-5cee-4ba7-a160-099c45f3cfe2 service cinder] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:36:56.320559 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d230a9b2-d852-406f-bb87-7633c45bd7c1 tempest-VolumeQuotaClassesTest-1619203118 tempest-VolumeQuotaClassesTest-1619203118-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85db2a953f74e578269be1476633b72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1219a34b406149b8931fb9f76f1ee82c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:36:56.321166 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d230a9b2-d852-406f-bb87-7633c45bd7c1 tempest-VolumeQuotaClassesTest-1619203118 tempest-VolumeQuotaClassesTest-1619203118-project-admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:36:56.321451 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d230a9b2-d852-406f-bb87-7633c45bd7c1 tempest-VolumeQuotaClassesTest-1619203118 tempest-VolumeQuotaClassesTest-1619203118-project-admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:36:56.321727 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d230a9b2-d852-406f-bb87-7633c45bd7c1 tempest-VolumeQuotaClassesTest-1619203118 tempest-VolumeQuotaClassesTest-1619203118-project-admin] PATH_INFO: `/v3/projects` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:36:56.322388 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d230a9b2-d852-406f-bb87-7633c45bd7c1 tempest-VolumeQuotaClassesTest-1619203118 tempest-VolumeQuotaClassesTest-1619203118-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:36:56.324280 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d230a9b2-d852-406f-bb87-7633c45bd7c1 tempest-VolumeQuotaClassesTest-1619203118 tempest-VolumeQuotaClassesTest-1619203118-project-admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:36:56.434781 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1392/2799] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:36:56 2022] POST /identity/v3/projects => generated 355 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:36:56.438874 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffbf9c4-9893-49cc-b599-5b135aba5e16 tempest-VolumeQuotaClassesTest-1619203118 tempest-VolumeQuotaClassesTest-1619203118-project-admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:36:56.448795 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffbf9c4-9893-49cc-b599-5b135aba5e16 tempest-VolumeQuotaClassesTest-1619203118 tempest-VolumeQuotaClassesTest-1619203118-project-admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:36:56.505558 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a35c653-1f5d-45ea-865d-961319e2c421 tempest-VolumeQuotaClassesTest-1619203118 tempest-VolumeQuotaClassesTest-1619203118-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85db2a953f74e578269be1476633b72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1219a34b406149b8931fb9f76f1ee82c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:36:56.506096 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a35c653-1f5d-45ea-865d-961319e2c421 tempest-VolumeQuotaClassesTest-1619203118 tempest-VolumeQuotaClassesTest-1619203118-project-admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:36:56.506410 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a35c653-1f5d-45ea-865d-961319e2c421 tempest-VolumeQuotaClassesTest-1619203118 tempest-VolumeQuotaClassesTest-1619203118-project-admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:36:56.506688 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a35c653-1f5d-45ea-865d-961319e2c421 tempest-VolumeQuotaClassesTest-1619203118 tempest-VolumeQuotaClassesTest-1619203118-project-admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:36:56.507323 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a35c653-1f5d-45ea-865d-961319e2c421 tempest-VolumeQuotaClassesTest-1619203118 tempest-VolumeQuotaClassesTest-1619203118-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:36:56.507630 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a35c653-1f5d-45ea-865d-961319e2c421 tempest-VolumeQuotaClassesTest-1619203118 tempest-VolumeQuotaClassesTest-1619203118-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:36:56.509206 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a35c653-1f5d-45ea-865d-961319e2c421 tempest-VolumeQuotaClassesTest-1619203118 tempest-VolumeQuotaClassesTest-1619203118-project-admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:36:56.520235 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1408/2800] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Aug 25 03:36:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:36:56.559470 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d230a9b2-d852-406f-bb87-7633c45bd7c1 tempest-VolumeQuotaClassesTest-1619203118 tempest-VolumeQuotaClassesTest-1619203118-project-admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:36:56.569857 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d230a9b2-d852-406f-bb87-7633c45bd7c1 tempest-VolumeQuotaClassesTest-1619203118 tempest-VolumeQuotaClassesTest-1619203118-project-admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:36:56.574716 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6584c262-eeda-46d7-a810-efaecbf75e44 tempest-VolumeQuotaClassesTest-1619203118 tempest-VolumeQuotaClassesTest-1619203118-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85db2a953f74e578269be1476633b72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1219a34b406149b8931fb9f76f1ee82c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:36:56.575352 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6584c262-eeda-46d7-a810-efaecbf75e44 tempest-VolumeQuotaClassesTest-1619203118 tempest-VolumeQuotaClassesTest-1619203118-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:36:56.575630 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6584c262-eeda-46d7-a810-efaecbf75e44 tempest-VolumeQuotaClassesTest-1619203118 tempest-VolumeQuotaClassesTest-1619203118-project-admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:36:56.575900 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6584c262-eeda-46d7-a810-efaecbf75e44 tempest-VolumeQuotaClassesTest-1619203118 tempest-VolumeQuotaClassesTest-1619203118-project-admin] PATH_INFO: `/v3/projects/165d7874b8444d558a8061533122a255` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:36:56.577105 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6584c262-eeda-46d7-a810-efaecbf75e44 tempest-VolumeQuotaClassesTest-1619203118 tempest-VolumeQuotaClassesTest-1619203118-project-admin] RBAC: Authorizing `identity:delete_project(project_id=165d7874b8444d558a8061533122a255)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:36:56.579061 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6584c262-eeda-46d7-a810-efaecbf75e44 tempest-VolumeQuotaClassesTest-1619203118 tempest-VolumeQuotaClassesTest-1619203118-project-admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:36:56.719375 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-6584c262-eeda-46d7-a810-efaecbf75e44 tempest-VolumeQuotaClassesTest-1619203118 tempest-VolumeQuotaClassesTest-1619203118-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '165d7874b8444d558a8061533122a255', 'request_id': 'req-6584c262-eeda-46d7-a810-efaecbf75e44'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:36:56.798994 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-6584c262-eeda-46d7-a810-efaecbf75e44 tempest-VolumeQuotaClassesTest-1619203118 tempest-VolumeQuotaClassesTest-1619203118-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '165d7874b8444d558a8061533122a255', 'request_id': 'req-6584c262-eeda-46d7-a810-efaecbf75e44'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:36:56.801281 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=87,keystone:INSERT=7,keystone:DELETE=6 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:36:56.805360 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-6584c262-eeda-46d7-a810-efaecbf75e44 tempest-VolumeQuotaClassesTest-1619203118 tempest-VolumeQuotaClassesTest-1619203118-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '165d7874b8444d558a8061533122a255', 'request_id': 'req-6584c262-eeda-46d7-a810-efaecbf75e44'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:36:56.817054 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-6584c262-eeda-46d7-a810-efaecbf75e44 tempest-VolumeQuotaClassesTest-1619203118 tempest-VolumeQuotaClassesTest-1619203118-project-admin] The token cache is being invalidate because project 165d7874b8444d558a8061533122a255 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81995) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:36:56.817345 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-6584c262-eeda-46d7-a810-efaecbf75e44 tempest-VolumeQuotaClassesTest-1619203118 tempest-VolumeQuotaClassesTest-1619203118-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:36:56.817994 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-6584c262-eeda-46d7-a810-efaecbf75e44 tempest-VolumeQuotaClassesTest-1619203118 tempest-VolumeQuotaClassesTest-1619203118-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:36:56.819585 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1393/2801] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:36:56 2022] DELETE /identity/v3/projects/165d7874b8444d558a8061533122a255 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:36:57.340932 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a35c653-1f5d-45ea-865d-961319e2c421 tempest-VolumeQuotaClassesTest-1619203118 tempest-VolumeQuotaClassesTest-1619203118-project-admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:36:57.342716 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-7a35c653-1f5d-45ea-865d-961319e2c421 tempest-VolumeQuotaClassesTest-1619203118 tempest-VolumeQuotaClassesTest-1619203118-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:36:57.407052 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a35c653-1f5d-45ea-865d-961319e2c421 tempest-VolumeQuotaClassesTest-1619203118 tempest-VolumeQuotaClassesTest-1619203118-project-admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:36:57.460643 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5dba2a-e6fb-4511-a5f6-d43400cdab84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:36:57.461197 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5dba2a-e6fb-4511-a5f6-d43400cdab84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:36:57.461471 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5dba2a-e6fb-4511-a5f6-d43400cdab84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:36:57.461740 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5dba2a-e6fb-4511-a5f6-d43400cdab84 admin admin] PATH_INFO: `/v3/users/a12495f4993244b6983a1ea4d385376c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:36:57.462862 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5dba2a-e6fb-4511-a5f6-d43400cdab84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a12495f4993244b6983a1ea4d385376c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:36:57.464752 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5dba2a-e6fb-4511-a5f6-d43400cdab84 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:36:57.577446 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:36:57.590006 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-af5dba2a-e6fb-4511-a5f6-d43400cdab84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a12495f4993244b6983a1ea4d385376c'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:36:57.673104 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-af5dba2a-e6fb-4511-a5f6-d43400cdab84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a12495f4993244b6983a1ea4d385376c'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:36:57.673758 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-af5dba2a-e6fb-4511-a5f6-d43400cdab84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a12495f4993244b6983a1ea4d385376c'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:36:57.681919 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-af5dba2a-e6fb-4511-a5f6-d43400cdab84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a12495f4993244b6983a1ea4d385376c'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:36:57.694121 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-af5dba2a-e6fb-4511-a5f6-d43400cdab84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a12495f4993244b6983a1ea4d385376c'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:36:57.699941 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1409/2802] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:36:57 2022] DELETE /identity/v3/users/a12495f4993244b6983a1ea4d385376c => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:36:57.704567 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076acd1d-db07-4f77-9dbf-fc58408e4fa9 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:36:57.705091 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076acd1d-db07-4f77-9dbf-fc58408e4fa9 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:36:57.705091 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076acd1d-db07-4f77-9dbf-fc58408e4fa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:36:57.723470 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-076acd1d-db07-4f77-9dbf-fc58408e4fa9 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:36:57.724660 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-076acd1d-db07-4f77-9dbf-fc58408e4fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:36:57.781875 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1394/2803] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:36:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:36:57.792131 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5dba2a-e6fb-4511-a5f6-d43400cdab84 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:36:57.793533 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-af5dba2a-e6fb-4511-a5f6-d43400cdab84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:36:57.851709 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5dba2a-e6fb-4511-a5f6-d43400cdab84 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:36:57.855456 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f8f2b9-868b-4b04-aef1-b64d643798f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:36:57.856064 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f8f2b9-868b-4b04-aef1-b64d643798f3 service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:36:57.856346 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f8f2b9-868b-4b04-aef1-b64d643798f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:36:57.856621 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f8f2b9-868b-4b04-aef1-b64d643798f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:36:57.866167 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f8f2b9-868b-4b04-aef1-b64d643798f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:36:57.872282 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f8f2b9-868b-4b04-aef1-b64d643798f3 service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:36:57.878507 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1410/2804] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:36:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:36:58.314820 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076acd1d-db07-4f77-9dbf-fc58408e4fa9 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:36:58.316258 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-076acd1d-db07-4f77-9dbf-fc58408e4fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:36:58.330568 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076acd1d-db07-4f77-9dbf-fc58408e4fa9 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:36:58.335956 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4162c8-fc5d-4612-a4ee-6c140aca146f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:36:58.337171 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4162c8-fc5d-4612-a4ee-6c140aca146f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:36:58.337647 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4162c8-fc5d-4612-a4ee-6c140aca146f admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:36:58.338189 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4162c8-fc5d-4612-a4ee-6c140aca146f admin admin] PATH_INFO: `/v3/projects/c171b9a284c24ff19b13626187c088b2` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:36:58.339398 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4162c8-fc5d-4612-a4ee-6c140aca146f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c171b9a284c24ff19b13626187c088b2)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:36:58.341382 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4162c8-fc5d-4612-a4ee-6c140aca146f admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:36:58.449806 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-be4162c8-fc5d-4612-a4ee-6c140aca146f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c171b9a284c24ff19b13626187c088b2', 'request_id': 'req-be4162c8-fc5d-4612-a4ee-6c140aca146f'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:36:58.518675 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-be4162c8-fc5d-4612-a4ee-6c140aca146f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c171b9a284c24ff19b13626187c088b2', 'request_id': 'req-be4162c8-fc5d-4612-a4ee-6c140aca146f'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:36:58.524735 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-be4162c8-fc5d-4612-a4ee-6c140aca146f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c171b9a284c24ff19b13626187c088b2', 'request_id': 'req-be4162c8-fc5d-4612-a4ee-6c140aca146f'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:36:58.535593 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-be4162c8-fc5d-4612-a4ee-6c140aca146f admin admin] The token cache is being invalidate because project c171b9a284c24ff19b13626187c088b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81995) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:36:58.535878 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-be4162c8-fc5d-4612-a4ee-6c140aca146f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:36:58.537153 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-be4162c8-fc5d-4612-a4ee-6c140aca146f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:36:58.538744 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1395/2805] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:36:58 2022] DELETE /identity/v3/projects/c171b9a284c24ff19b13626187c088b2 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:36:58.542997 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f8f2b9-868b-4b04-aef1-b64d643798f3 service neutron] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:36:58.552674 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f8f2b9-868b-4b04-aef1-b64d643798f3 service neutron] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:36:58.611410 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e10fb2-fef4-4b9a-8890-c03ad88dcb04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:36:58.612007 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e10fb2-fef4-4b9a-8890-c03ad88dcb04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:36:58.612403 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e10fb2-fef4-4b9a-8890-c03ad88dcb04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:36:58.612664 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e10fb2-fef4-4b9a-8890-c03ad88dcb04 admin admin] PATH_INFO: `/v3/users/d85db2a953f74e578269be1476633b72` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:36:58.613758 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e10fb2-fef4-4b9a-8890-c03ad88dcb04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d85db2a953f74e578269be1476633b72)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:36:58.615859 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e10fb2-fef4-4b9a-8890-c03ad88dcb04 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:36:58.743354 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-d0e10fb2-fef4-4b9a-8890-c03ad88dcb04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd85db2a953f74e578269be1476633b72'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:36:58.804425 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-d0e10fb2-fef4-4b9a-8890-c03ad88dcb04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd85db2a953f74e578269be1476633b72'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:36:58.805048 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-d0e10fb2-fef4-4b9a-8890-c03ad88dcb04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd85db2a953f74e578269be1476633b72'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:36:58.806596 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:36:58.812824 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-d0e10fb2-fef4-4b9a-8890-c03ad88dcb04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd85db2a953f74e578269be1476633b72'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:36:58.824972 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-d0e10fb2-fef4-4b9a-8890-c03ad88dcb04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd85db2a953f74e578269be1476633b72'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:36:58.831140 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1411/2806] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:36:58 2022] DELETE /identity/v3/users/d85db2a953f74e578269be1476633b72 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:36:59.205748 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4162c8-fc5d-4612-a4ee-6c140aca146f admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:36:59.207214 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-be4162c8-fc5d-4612-a4ee-6c140aca146f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:36:59.225861 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4162c8-fc5d-4612-a4ee-6c140aca146f admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:36:59.297773 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34507c8b-7f6d-41b8-8b09-2969ca46f7f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:36:59.298447 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34507c8b-7f6d-41b8-8b09-2969ca46f7f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:36:59.298726 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34507c8b-7f6d-41b8-8b09-2969ca46f7f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:36:59.299001 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34507c8b-7f6d-41b8-8b09-2969ca46f7f8 admin admin] PATH_INFO: `/v3/projects/1219a34b406149b8931fb9f76f1ee82c` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:36:59.300136 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34507c8b-7f6d-41b8-8b09-2969ca46f7f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1219a34b406149b8931fb9f76f1ee82c)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:36:59.302070 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34507c8b-7f6d-41b8-8b09-2969ca46f7f8 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:36:59.313805 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:36:59.386855 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-34507c8b-7f6d-41b8-8b09-2969ca46f7f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1219a34b406149b8931fb9f76f1ee82c', 'request_id': 'req-34507c8b-7f6d-41b8-8b09-2969ca46f7f8'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:36:59.461413 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-34507c8b-7f6d-41b8-8b09-2969ca46f7f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1219a34b406149b8931fb9f76f1ee82c', 'request_id': 'req-34507c8b-7f6d-41b8-8b09-2969ca46f7f8'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:36:59.467186 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-34507c8b-7f6d-41b8-8b09-2969ca46f7f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1219a34b406149b8931fb9f76f1ee82c', 'request_id': 'req-34507c8b-7f6d-41b8-8b09-2969ca46f7f8'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:36:59.478699 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-34507c8b-7f6d-41b8-8b09-2969ca46f7f8 admin admin] The token cache is being invalidate because project 1219a34b406149b8931fb9f76f1ee82c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81995) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:36:59.478992 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-34507c8b-7f6d-41b8-8b09-2969ca46f7f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:36:59.479621 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-34507c8b-7f6d-41b8-8b09-2969ca46f7f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:36:59.481120 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1396/2807] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:36:59 2022] DELETE /identity/v3/projects/1219a34b406149b8931fb9f76f1ee82c => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:36:59.486083 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9099ae8-e8fb-4b14-bfdd-f8cef5bd53f1 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:36:59.486400 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9099ae8-e8fb-4b14-bfdd-f8cef5bd53f1 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:36:59.486708 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9099ae8-e8fb-4b14-bfdd-f8cef5bd53f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:36:59.506248 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-b9099ae8-e8fb-4b14-bfdd-f8cef5bd53f1 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:36:59.507463 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-b9099ae8-e8fb-4b14-bfdd-f8cef5bd53f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:36:59.566507 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1412/2808] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:36:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:36:59.579545 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b87c280-6883-4165-8522-c1a87964c15e None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:36:59.579886 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b87c280-6883-4165-8522-c1a87964c15e None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:36:59.580207 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b87c280-6883-4165-8522-c1a87964c15e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:36:59.600622 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-8b87c280-6883-4165-8522-c1a87964c15e None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:36:59.601874 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-8b87c280-6883-4165-8522-c1a87964c15e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:36:59.609741 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1397/2809] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:36:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:36:59.613934 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9099ae8-e8fb-4b14-bfdd-f8cef5bd53f1 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:36:59.624242 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9099ae8-e8fb-4b14-bfdd-f8cef5bd53f1 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:36:59.628815 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c7b418-024c-4d46-930e-bcbea4877caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:36:59.629414 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c7b418-024c-4d46-930e-bcbea4877caf admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:36:59.629687 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c7b418-024c-4d46-930e-bcbea4877caf admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:36:59.629966 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c7b418-024c-4d46-930e-bcbea4877caf admin admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:36:59.630651 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c7b418-024c-4d46-930e-bcbea4877caf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:36:59.630956 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c7b418-024c-4d46-930e-bcbea4877caf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:36:59.632556 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c7b418-024c-4d46-930e-bcbea4877caf admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:36:59.644121 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1413/2810] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Aug 25 03:36:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:36:59.652524 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b87c280-6883-4165-8522-c1a87964c15e None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:36:59.662631 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b87c280-6883-4165-8522-c1a87964c15e None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:36:59.667335 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2479769f-3799-4ab4-bd57-b09424394547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:36:59.667936 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2479769f-3799-4ab4-bd57-b09424394547 admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:36:59.668215 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2479769f-3799-4ab4-bd57-b09424394547 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:36:59.668538 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2479769f-3799-4ab4-bd57-b09424394547 admin admin] PATH_INFO: `/v3/projects` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:36:59.669149 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2479769f-3799-4ab4-bd57-b09424394547 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:36:59.671092 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2479769f-3799-4ab4-bd57-b09424394547 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:36:59.733345 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1398/2811] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:36:59 2022] POST /identity/v3/projects => generated 385 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:36:59.735317 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c7b418-024c-4d46-930e-bcbea4877caf admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:36:59.745511 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c7b418-024c-4d46-930e-bcbea4877caf admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:36:59.800586 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28012cb6-540e-4145-b7e4-47270cd4fbbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:36:59.801192 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28012cb6-540e-4145-b7e4-47270cd4fbbf admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:36:59.801472 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28012cb6-540e-4145-b7e4-47270cd4fbbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:36:59.801751 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28012cb6-540e-4145-b7e4-47270cd4fbbf admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:36:59.802441 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28012cb6-540e-4145-b7e4-47270cd4fbbf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:36:59.804351 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28012cb6-540e-4145-b7e4-47270cd4fbbf admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:36:59.892266 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1414/2812] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:36:59 2022] POST /identity/v3/users => generated 293 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:36:59.896522 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2479769f-3799-4ab4-bd57-b09424394547 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:36:59.907356 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2479769f-3799-4ab4-bd57-b09424394547 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:36:59.912832 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8885156-fbb7-42cc-8733-a1ed1aaf86da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:36:59.913429 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8885156-fbb7-42cc-8733-a1ed1aaf86da admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:36:59.913709 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8885156-fbb7-42cc-8733-a1ed1aaf86da admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:36:59.913984 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8885156-fbb7-42cc-8733-a1ed1aaf86da admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:36:59.914697 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8885156-fbb7-42cc-8733-a1ed1aaf86da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:36:59.916290 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8885156-fbb7-42cc-8733-a1ed1aaf86da admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:36:59.926057 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1399/2813] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:36:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:36:59.930066 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28012cb6-540e-4145-b7e4-47270cd4fbbf admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:36:59.940178 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28012cb6-540e-4145-b7e4-47270cd4fbbf admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:36:59.944765 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1441f2f-fc49-489d-a998-df38222602dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:36:59.945355 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1441f2f-fc49-489d-a998-df38222602dc admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:36:59.945635 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1441f2f-fc49-489d-a998-df38222602dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:36:59.945908 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1441f2f-fc49-489d-a998-df38222602dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:36:59.946580 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1441f2f-fc49-489d-a998-df38222602dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:36:59.948169 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1441f2f-fc49-489d-a998-df38222602dc admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:36:59.959111 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1415/2814] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:36:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:36:59.963113 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8885156-fbb7-42cc-8733-a1ed1aaf86da admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:36:59.972866 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8885156-fbb7-42cc-8733-a1ed1aaf86da admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:36:59.978126 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c306312c-8775-4a02-8f4a-4f4e91dcde7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:36:59.978796 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c306312c-8775-4a02-8f4a-4f4e91dcde7e admin admin] REQUEST_METHOD: `PUT` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:36:59.979075 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c306312c-8775-4a02-8f4a-4f4e91dcde7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:36:59.979345 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c306312c-8775-4a02-8f4a-4f4e91dcde7e admin admin] PATH_INFO: `/v3/projects/93c9797ef55f49d49a3b55369cff5bc5/users/1e3ef2098a024f0390d6baf55908b15e/roles/3255f77192524a16bc405173106871fc` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:36:59.993544 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c306312c-8775-4a02-8f4a-4f4e91dcde7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=93c9797ef55f49d49a3b55369cff5bc5, user_id=1e3ef2098a024f0390d6baf55908b15e, role_id=3255f77192524a16bc405173106871fc)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:36:59.996146 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c306312c-8775-4a02-8f4a-4f4e91dcde7e admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:37:00.085257 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1400/2815] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:36:59 2022] PUT /identity/v3/projects/93c9797ef55f49d49a3b55369cff5bc5/users/1e3ef2098a024f0390d6baf55908b15e/roles/3255f77192524a16bc405173106871fc => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:37:00.096972 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1cffbc-c914-4f99-965f-a35fdf6b1b3d None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:00.097257 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1cffbc-c914-4f99-965f-a35fdf6b1b3d None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:00.097533 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1cffbc-c914-4f99-965f-a35fdf6b1b3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:00.124506 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-0c1cffbc-c914-4f99-965f-a35fdf6b1b3d None None] MFA Rules not processed for user `1e3ef2098a024f0390d6baf55908b15e`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:37:00.125722 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-0c1cffbc-c914-4f99-965f-a35fdf6b1b3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:37:00.153243 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:37:00.190475 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1416/2816] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:37:00 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:37:00.195642 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c306312c-8775-4a02-8f4a-4f4e91dcde7e admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:37:00.255927 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c306312c-8775-4a02-8f4a-4f4e91dcde7e admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:37:00.260556 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0996f1bd-6477-49e0-b5c1-6103cdd6687b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:37:00.261158 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0996f1bd-6477-49e0-b5c1-6103cdd6687b admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:00.261438 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0996f1bd-6477-49e0-b5c1-6103cdd6687b admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:00.261712 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0996f1bd-6477-49e0-b5c1-6103cdd6687b admin admin] PATH_INFO: `/v3/projects` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:00.262376 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0996f1bd-6477-49e0-b5c1-6103cdd6687b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:37:00.264271 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0996f1bd-6477-49e0-b5c1-6103cdd6687b admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:37:00.373329 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1401/2817] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:37:00 2022] POST /identity/v3/projects => generated 385 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:37:00.375623 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1cffbc-c914-4f99-965f-a35fdf6b1b3d None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:37:00.386817 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1cffbc-c914-4f99-965f-a35fdf6b1b3d None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:37:00.443635 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696b3078-4bcc-4668-940c-59c0b78d510a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:37:00.444348 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696b3078-4bcc-4668-940c-59c0b78d510a admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:00.444771 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696b3078-4bcc-4668-940c-59c0b78d510a admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:00.445120 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696b3078-4bcc-4668-940c-59c0b78d510a admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:00.445758 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696b3078-4bcc-4668-940c-59c0b78d510a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:37:00.447968 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696b3078-4bcc-4668-940c-59c0b78d510a admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:37:00.513339 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1417/2818] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:37:00 2022] POST /identity/v3/users => generated 299 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:37:00.517681 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0996f1bd-6477-49e0-b5c1-6103cdd6687b admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:37:00.527974 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0996f1bd-6477-49e0-b5c1-6103cdd6687b admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:37:00.532576 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9098b1d-a4c1-4a0c-9027-cb38f57fa309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:37:00.533184 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9098b1d-a4c1-4a0c-9027-cb38f57fa309 admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:00.533512 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9098b1d-a4c1-4a0c-9027-cb38f57fa309 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:00.533793 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9098b1d-a4c1-4a0c-9027-cb38f57fa309 admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:00.534471 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9098b1d-a4c1-4a0c-9027-cb38f57fa309 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:37:00.536129 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9098b1d-a4c1-4a0c-9027-cb38f57fa309 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:37:00.547579 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1402/2819] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:37:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:37:00.551564 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696b3078-4bcc-4668-940c-59c0b78d510a admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:37:00.563025 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696b3078-4bcc-4668-940c-59c0b78d510a admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:37:00.568607 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12b039c-cb7c-4875-ac3d-d0e7caa2bea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:37:00.569245 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12b039c-cb7c-4875-ac3d-d0e7caa2bea1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:00.569510 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12b039c-cb7c-4875-ac3d-d0e7caa2bea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:00.569780 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12b039c-cb7c-4875-ac3d-d0e7caa2bea1 admin admin] PATH_INFO: `/v3/projects/b4b433c57a7e4dafac0d12ad17c89b24/users/5436b22dfd9641a68932b8f07d4ef36f/roles/62f19ea272b447ccaab1576cd9bf4fd9` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:00.584678 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12b039c-cb7c-4875-ac3d-d0e7caa2bea1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4b433c57a7e4dafac0d12ad17c89b24, user_id=5436b22dfd9641a68932b8f07d4ef36f, role_id=62f19ea272b447ccaab1576cd9bf4fd9)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:37:00.587391 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12b039c-cb7c-4875-ac3d-d0e7caa2bea1 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:37:00.656297 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1418/2820] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:37:00 2022] PUT /identity/v3/projects/b4b433c57a7e4dafac0d12ad17c89b24/users/5436b22dfd9641a68932b8f07d4ef36f/roles/62f19ea272b447ccaab1576cd9bf4fd9 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:37:00.660508 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9098b1d-a4c1-4a0c-9027-cb38f57fa309 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:37:00.670793 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9098b1d-a4c1-4a0c-9027-cb38f57fa309 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:37:00.679351 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:37:00.726932 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f9053a-4ed1-4dc5-8797-956ac8f1edad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:37:00.727534 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f9053a-4ed1-4dc5-8797-956ac8f1edad admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:00.727813 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f9053a-4ed1-4dc5-8797-956ac8f1edad admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:00.728104 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f9053a-4ed1-4dc5-8797-956ac8f1edad admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:00.728738 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f9053a-4ed1-4dc5-8797-956ac8f1edad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:37:00.730766 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f9053a-4ed1-4dc5-8797-956ac8f1edad admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:37:00.741193 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1403/2821] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:37:00 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:37:00.745426 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12b039c-cb7c-4875-ac3d-d0e7caa2bea1 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:37:00.756550 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12b039c-cb7c-4875-ac3d-d0e7caa2bea1 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:37:00.761436 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c689830-60ce-4c7e-9422-93c12913d641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:37:00.762089 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c689830-60ce-4c7e-9422-93c12913d641 admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:00.762408 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c689830-60ce-4c7e-9422-93c12913d641 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:00.762690 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c689830-60ce-4c7e-9422-93c12913d641 admin admin] PATH_INFO: `/v3/projects/b4b433c57a7e4dafac0d12ad17c89b24/users/5436b22dfd9641a68932b8f07d4ef36f/roles/62f19ea272b447ccaab1576cd9bf4fd9` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:00.771501 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c689830-60ce-4c7e-9422-93c12913d641 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4b433c57a7e4dafac0d12ad17c89b24, user_id=5436b22dfd9641a68932b8f07d4ef36f, role_id=62f19ea272b447ccaab1576cd9bf4fd9)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:37:00.771501 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c689830-60ce-4c7e-9422-93c12913d641 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:37:00.777647 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1419/2822] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:37:00 2022] PUT /identity/v3/projects/b4b433c57a7e4dafac0d12ad17c89b24/users/5436b22dfd9641a68932b8f07d4ef36f/roles/62f19ea272b447ccaab1576cd9bf4fd9 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:37:00.788256 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ee1cc5-1b94-45ca-8443-4bc2953feef4 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:00.788581 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ee1cc5-1b94-45ca-8443-4bc2953feef4 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:00.788890 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ee1cc5-1b94-45ca-8443-4bc2953feef4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:00.817209 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-69ee1cc5-1b94-45ca-8443-4bc2953feef4 None None] MFA Rules not processed for user `5436b22dfd9641a68932b8f07d4ef36f`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:37:00.818456 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-69ee1cc5-1b94-45ca-8443-4bc2953feef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:37:00.884653 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1404/2823] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:37:00 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:37:00.895463 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c689830-60ce-4c7e-9422-93c12913d641 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:37:00.897157 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-0c689830-60ce-4c7e-9422-93c12913d641 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:37:00.953367 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c689830-60ce-4c7e-9422-93c12913d641 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:37:00.956970 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f8250d-adc3-4a0c-a810-f26554b4d991 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60193ba9e1c14df3bc9eebcc6f28f2d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:37:00.957568 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f8250d-adc3-4a0c-a810-f26554b4d991 service cinder] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:00.957846 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f8250d-adc3-4a0c-a810-f26554b4d991 service cinder] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:00.958121 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f8250d-adc3-4a0c-a810-f26554b4d991 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:00.966906 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f8250d-adc3-4a0c-a810-f26554b4d991 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:37:00.968324 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f8250d-adc3-4a0c-a810-f26554b4d991 service cinder] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:37:00.978014 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1420/2824] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:37:00 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:37:01.112800 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ee1cc5-1b94-45ca-8443-4bc2953feef4 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:37:01.173643 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ee1cc5-1b94-45ca-8443-4bc2953feef4 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:37:01.178300 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8892aac3-5456-4746-8337-b7e1e11625c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:37:01.178931 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8892aac3-5456-4746-8337-b7e1e11625c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:01.179212 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8892aac3-5456-4746-8337-b7e1e11625c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:01.179484 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8892aac3-5456-4746-8337-b7e1e11625c6 admin admin] PATH_INFO: `/v3/users/1e3ef2098a024f0390d6baf55908b15e` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:01.180580 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8892aac3-5456-4746-8337-b7e1e11625c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e3ef2098a024f0390d6baf55908b15e)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:37:01.182785 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8892aac3-5456-4746-8337-b7e1e11625c6 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:37:01.340188 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:37:01.348760 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-8892aac3-5456-4746-8337-b7e1e11625c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e3ef2098a024f0390d6baf55908b15e'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:37:01.443883 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-8892aac3-5456-4746-8337-b7e1e11625c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e3ef2098a024f0390d6baf55908b15e'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:37:01.444484 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-8892aac3-5456-4746-8337-b7e1e11625c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e3ef2098a024f0390d6baf55908b15e'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:37:01.445076 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-8892aac3-5456-4746-8337-b7e1e11625c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e3ef2098a024f0390d6baf55908b15e'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:37:01.457701 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-8892aac3-5456-4746-8337-b7e1e11625c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e3ef2098a024f0390d6baf55908b15e'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:37:01.470318 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1405/2825] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:37:01 2022] DELETE /identity/v3/users/1e3ef2098a024f0390d6baf55908b15e => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:37:01.475506 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaa4cee-babe-46c0-9b43-c72275388931 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:01.475795 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaa4cee-babe-46c0-9b43-c72275388931 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:01.476075 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaa4cee-babe-46c0-9b43-c72275388931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:01.494656 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-8eaa4cee-babe-46c0-9b43-c72275388931 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:37:01.495849 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-8eaa4cee-babe-46c0-9b43-c72275388931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:37:01.511963 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:37:01.553763 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1421/2826] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:37:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:37:01.563829 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8892aac3-5456-4746-8337-b7e1e11625c6 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:37:01.565215 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-8892aac3-5456-4746-8337-b7e1e11625c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:37:01.628027 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8892aac3-5456-4746-8337-b7e1e11625c6 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:37:01.632417 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abb7d4a-aa19-453c-98d2-eb388a94cb05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:37:01.633058 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abb7d4a-aa19-453c-98d2-eb388a94cb05 service neutron] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:01.633372 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abb7d4a-aa19-453c-98d2-eb388a94cb05 service neutron] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:01.633712 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abb7d4a-aa19-453c-98d2-eb388a94cb05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:01.643735 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abb7d4a-aa19-453c-98d2-eb388a94cb05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:37:01.645184 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abb7d4a-aa19-453c-98d2-eb388a94cb05 service neutron] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:37:01.655809 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1406/2827] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:37:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 25 03:37:02.040819 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaa4cee-babe-46c0-9b43-c72275388931 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:37:02.042409 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-8eaa4cee-babe-46c0-9b43-c72275388931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:37:02.055389 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaa4cee-babe-46c0-9b43-c72275388931 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:37:02.060453 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496dd440-d222-440f-832e-db452dc38745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:37:02.061088 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496dd440-d222-440f-832e-db452dc38745 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:02.061366 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496dd440-d222-440f-832e-db452dc38745 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:02.061640 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496dd440-d222-440f-832e-db452dc38745 admin admin] PATH_INFO: `/v3/projects/93c9797ef55f49d49a3b55369cff5bc5` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:02.062800 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496dd440-d222-440f-832e-db452dc38745 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93c9797ef55f49d49a3b55369cff5bc5)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:37:02.064690 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496dd440-d222-440f-832e-db452dc38745 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:37:02.152171 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-496dd440-d222-440f-832e-db452dc38745 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93c9797ef55f49d49a3b55369cff5bc5', 'request_id': 'req-496dd440-d222-440f-832e-db452dc38745'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:37:02.198109 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-496dd440-d222-440f-832e-db452dc38745 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93c9797ef55f49d49a3b55369cff5bc5', 'request_id': 'req-496dd440-d222-440f-832e-db452dc38745'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:37:02.203755 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-496dd440-d222-440f-832e-db452dc38745 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93c9797ef55f49d49a3b55369cff5bc5', 'request_id': 'req-496dd440-d222-440f-832e-db452dc38745'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:37:02.214761 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-496dd440-d222-440f-832e-db452dc38745 admin admin] The token cache is being invalidate because project 93c9797ef55f49d49a3b55369cff5bc5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:37:02.215048 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-496dd440-d222-440f-832e-db452dc38745 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:37:02.215680 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-496dd440-d222-440f-832e-db452dc38745 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:37:02.217203 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1422/2828] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:37:02 2022] DELETE /identity/v3/projects/93c9797ef55f49d49a3b55369cff5bc5 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:37:02.221202 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abb7d4a-aa19-453c-98d2-eb388a94cb05 service neutron] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:37:02.231093 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abb7d4a-aa19-453c-98d2-eb388a94cb05 service neutron] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:37:02.291340 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77aea57-551b-42e3-83a0-b0c9b74c407f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:37:02.291974 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77aea57-551b-42e3-83a0-b0c9b74c407f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:02.292255 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77aea57-551b-42e3-83a0-b0c9b74c407f admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:02.292533 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77aea57-551b-42e3-83a0-b0c9b74c407f admin admin] PATH_INFO: `/v3/users/5436b22dfd9641a68932b8f07d4ef36f` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:02.293726 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77aea57-551b-42e3-83a0-b0c9b74c407f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5436b22dfd9641a68932b8f07d4ef36f)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:37:02.295683 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77aea57-551b-42e3-83a0-b0c9b74c407f admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:37:02.468626 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-c77aea57-551b-42e3-83a0-b0c9b74c407f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5436b22dfd9641a68932b8f07d4ef36f'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:37:02.512520 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-c77aea57-551b-42e3-83a0-b0c9b74c407f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5436b22dfd9641a68932b8f07d4ef36f'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:37:02.513145 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-c77aea57-551b-42e3-83a0-b0c9b74c407f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5436b22dfd9641a68932b8f07d4ef36f'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:37:02.513820 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-c77aea57-551b-42e3-83a0-b0c9b74c407f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5436b22dfd9641a68932b8f07d4ef36f'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:37:02.515860 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:37:02.526440 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-c77aea57-551b-42e3-83a0-b0c9b74c407f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5436b22dfd9641a68932b8f07d4ef36f'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:37:02.538887 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1407/2829] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:37:02 2022] DELETE /identity/v3/users/5436b22dfd9641a68932b8f07d4ef36f => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:37:02.902778 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496dd440-d222-440f-832e-db452dc38745 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:37:02.904235 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-496dd440-d222-440f-832e-db452dc38745 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:37:02.920575 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496dd440-d222-440f-832e-db452dc38745 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:37:02.977414 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17741ef-d457-4418-8ed6-1d56de1dfea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:37:02.978053 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17741ef-d457-4418-8ed6-1d56de1dfea4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:02.978370 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17741ef-d457-4418-8ed6-1d56de1dfea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:02.978650 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17741ef-d457-4418-8ed6-1d56de1dfea4 admin admin] PATH_INFO: `/v3/projects/b4b433c57a7e4dafac0d12ad17c89b24` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:02.979814 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17741ef-d457-4418-8ed6-1d56de1dfea4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4b433c57a7e4dafac0d12ad17c89b24)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:37:02.981672 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17741ef-d457-4418-8ed6-1d56de1dfea4 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:37:03.076740 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:37:03.078065 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-d17741ef-d457-4418-8ed6-1d56de1dfea4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4b433c57a7e4dafac0d12ad17c89b24', 'request_id': 'req-d17741ef-d457-4418-8ed6-1d56de1dfea4'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:37:03.164436 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-d17741ef-d457-4418-8ed6-1d56de1dfea4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4b433c57a7e4dafac0d12ad17c89b24', 'request_id': 'req-d17741ef-d457-4418-8ed6-1d56de1dfea4'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:37:03.170183 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-d17741ef-d457-4418-8ed6-1d56de1dfea4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4b433c57a7e4dafac0d12ad17c89b24', 'request_id': 'req-d17741ef-d457-4418-8ed6-1d56de1dfea4'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:37:03.181707 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-d17741ef-d457-4418-8ed6-1d56de1dfea4 admin admin] The token cache is being invalidate because project b4b433c57a7e4dafac0d12ad17c89b24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:37:03.181996 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-d17741ef-d457-4418-8ed6-1d56de1dfea4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:37:03.182652 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-d17741ef-d457-4418-8ed6-1d56de1dfea4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:37:03.184141 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1423/2830] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:37:02 2022] DELETE /identity/v3/projects/b4b433c57a7e4dafac0d12ad17c89b24 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:37:03.189004 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348ac0c7-69f0-46fa-a5b9-594a16527844 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:03.189293 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348ac0c7-69f0-46fa-a5b9-594a16527844 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:03.189567 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348ac0c7-69f0-46fa-a5b9-594a16527844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:03.209262 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-348ac0c7-69f0-46fa-a5b9-594a16527844 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:37:03.210500 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-348ac0c7-69f0-46fa-a5b9-594a16527844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:37:03.269208 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1408/2831] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:37:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:37:03.277208 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fff146-9e0e-49ad-81fb-5901f10f0b69 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:03.277534 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fff146-9e0e-49ad-81fb-5901f10f0b69 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:03.277815 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fff146-9e0e-49ad-81fb-5901f10f0b69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:03.297493 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-29fff146-9e0e-49ad-81fb-5901f10f0b69 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:37:03.298721 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-29fff146-9e0e-49ad-81fb-5901f10f0b69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:37:03.307421 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1424/2832] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:37:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:37:03.311516 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348ac0c7-69f0-46fa-a5b9-594a16527844 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:37:03.321865 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348ac0c7-69f0-46fa-a5b9-594a16527844 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:37:03.326533 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf22589-7d4c-4f7e-8147-623934b2e8ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:37:03.327195 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf22589-7d4c-4f7e-8147-623934b2e8ec admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:03.327476 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf22589-7d4c-4f7e-8147-623934b2e8ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:03.327748 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf22589-7d4c-4f7e-8147-623934b2e8ec admin admin] PATH_INFO: `/v3/domains` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:03.328383 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf22589-7d4c-4f7e-8147-623934b2e8ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81995) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:37:03.328689 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf22589-7d4c-4f7e-8147-623934b2e8ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:37:03.330368 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf22589-7d4c-4f7e-8147-623934b2e8ec admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:37:03.342084 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1409/2833] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Aug 25 03:37:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:37:03.346347 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fff146-9e0e-49ad-81fb-5901f10f0b69 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:37:03.355719 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fff146-9e0e-49ad-81fb-5901f10f0b69 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:37:03.360347 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06231984-dbcf-44ed-b582-ee585e21c6f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:37:03.360943 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06231984-dbcf-44ed-b582-ee585e21c6f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:03.361225 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06231984-dbcf-44ed-b582-ee585e21c6f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:03.361509 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06231984-dbcf-44ed-b582-ee585e21c6f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:03.362133 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06231984-dbcf-44ed-b582-ee585e21c6f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:37:03.364132 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06231984-dbcf-44ed-b582-ee585e21c6f6 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:37:03.451439 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1425/2834] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:37:03 2022] POST /identity/v3/projects => generated 381 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:37:03.455560 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf22589-7d4c-4f7e-8147-623934b2e8ec admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:37:03.466351 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf22589-7d4c-4f7e-8147-623934b2e8ec admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:37:03.524012 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e078a3-6379-4790-bf90-bd0277148581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:37:03.524649 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e078a3-6379-4790-bf90-bd0277148581 admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:03.524817 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e078a3-6379-4790-bf90-bd0277148581 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:03.525089 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e078a3-6379-4790-bf90-bd0277148581 admin admin] PATH_INFO: `/v3/users` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:03.525721 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e078a3-6379-4790-bf90-bd0277148581 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:37:03.527786 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e078a3-6379-4790-bf90-bd0277148581 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:37:03.604937 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1410/2835] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:37:03 2022] POST /identity/v3/users => generated 291 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:37:03.608852 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06231984-dbcf-44ed-b582-ee585e21c6f6 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:37:03.619416 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06231984-dbcf-44ed-b582-ee585e21c6f6 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:37:03.624165 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41647ad4-2928-46f8-b36a-3424fc8cd4ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:37:03.624767 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41647ad4-2928-46f8-b36a-3424fc8cd4ee admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:03.625046 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41647ad4-2928-46f8-b36a-3424fc8cd4ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:03.625319 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41647ad4-2928-46f8-b36a-3424fc8cd4ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:03.626005 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41647ad4-2928-46f8-b36a-3424fc8cd4ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:37:03.627760 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41647ad4-2928-46f8-b36a-3424fc8cd4ee admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:37:03.639023 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1426/2836] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:37:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:37:03.642881 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e078a3-6379-4790-bf90-bd0277148581 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:37:03.653688 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e078a3-6379-4790-bf90-bd0277148581 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:37:03.658327 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4fcdd1-1434-46bb-8f54-d242dd7eab49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:37:03.658923 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4fcdd1-1434-46bb-8f54-d242dd7eab49 admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:03.659206 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4fcdd1-1434-46bb-8f54-d242dd7eab49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:03.659486 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4fcdd1-1434-46bb-8f54-d242dd7eab49 admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:03.660143 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4fcdd1-1434-46bb-8f54-d242dd7eab49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:37:03.661732 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4fcdd1-1434-46bb-8f54-d242dd7eab49 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:37:03.671659 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1411/2837] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:37:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:37:03.675672 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41647ad4-2928-46f8-b36a-3424fc8cd4ee admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:37:03.685864 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41647ad4-2928-46f8-b36a-3424fc8cd4ee admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:37:03.690450 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f307a3-2ece-42d5-b261-4e7c25e850eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:37:03.691091 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f307a3-2ece-42d5-b261-4e7c25e850eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:03.691374 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f307a3-2ece-42d5-b261-4e7c25e850eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:03.691649 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f307a3-2ece-42d5-b261-4e7c25e850eb admin admin] PATH_INFO: `/v3/projects/6cd7954fca7c45a184567c0a1b091bf9/users/201d76cf02a04bb38da134d6bd68d92a/roles/3255f77192524a16bc405173106871fc` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:03.707097 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f307a3-2ece-42d5-b261-4e7c25e850eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cd7954fca7c45a184567c0a1b091bf9, user_id=201d76cf02a04bb38da134d6bd68d92a, role_id=3255f77192524a16bc405173106871fc)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:37:03.710359 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f307a3-2ece-42d5-b261-4e7c25e850eb admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:37:03.810508 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1427/2838] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:37:03 2022] PUT /identity/v3/projects/6cd7954fca7c45a184567c0a1b091bf9/users/201d76cf02a04bb38da134d6bd68d92a/roles/3255f77192524a16bc405173106871fc => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:37:03.822135 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371fe77c-a772-41a0-b971-47750db039a1 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:03.822455 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371fe77c-a772-41a0-b971-47750db039a1 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:03.822731 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371fe77c-a772-41a0-b971-47750db039a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:03.850105 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-371fe77c-a772-41a0-b971-47750db039a1 None None] MFA Rules not processed for user `201d76cf02a04bb38da134d6bd68d92a`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:37:03.851355 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-371fe77c-a772-41a0-b971-47750db039a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:37:03.879610 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:37:03.914599 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1412/2839] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:37:03 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:37:03.918993 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f307a3-2ece-42d5-b261-4e7c25e850eb admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:37:03.981479 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f307a3-2ece-42d5-b261-4e7c25e850eb admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:37:03.986134 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578d9e84-9476-49f2-89da-550ba9c28e39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:37:03.986777 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d9e84-9476-49f2-89da-550ba9c28e39 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:03.987108 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d9e84-9476-49f2-89da-550ba9c28e39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:03.987395 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d9e84-9476-49f2-89da-550ba9c28e39 admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:03.988027 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578d9e84-9476-49f2-89da-550ba9c28e39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:37:03.990480 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578d9e84-9476-49f2-89da-550ba9c28e39 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:37:04.074564 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1428/2840] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:37:03 2022] POST /identity/v3/projects => generated 381 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:37:04.078569 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371fe77c-a772-41a0-b971-47750db039a1 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:37:04.089702 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371fe77c-a772-41a0-b971-47750db039a1 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:37:04.145319 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd8029b-7759-43ed-b547-2b7587835bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:37:04.145928 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd8029b-7759-43ed-b547-2b7587835bbe admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:04.146211 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd8029b-7759-43ed-b547-2b7587835bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:04.146520 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd8029b-7759-43ed-b547-2b7587835bbe admin admin] PATH_INFO: `/v3/users` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:04.147151 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd8029b-7759-43ed-b547-2b7587835bbe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:37:04.149064 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd8029b-7759-43ed-b547-2b7587835bbe admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:37:04.250513 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1413/2841] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:37:04 2022] POST /identity/v3/users => generated 297 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:37:04.254519 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578d9e84-9476-49f2-89da-550ba9c28e39 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:37:04.265444 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578d9e84-9476-49f2-89da-550ba9c28e39 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:37:04.270142 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cffe76-3b57-4945-a7a1-cc5c48ba046e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:37:04.270782 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cffe76-3b57-4945-a7a1-cc5c48ba046e admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:04.271065 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cffe76-3b57-4945-a7a1-cc5c48ba046e admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:04.271343 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cffe76-3b57-4945-a7a1-cc5c48ba046e admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:04.272017 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cffe76-3b57-4945-a7a1-cc5c48ba046e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:37:04.273608 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cffe76-3b57-4945-a7a1-cc5c48ba046e admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:37:04.284063 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1429/2842] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:37:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:37:04.288767 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd8029b-7759-43ed-b547-2b7587835bbe admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:37:04.298636 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd8029b-7759-43ed-b547-2b7587835bbe admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:37:04.303922 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53840ee8-ffc3-4e3e-87db-33c2766a6056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:37:04.304646 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53840ee8-ffc3-4e3e-87db-33c2766a6056 admin admin] REQUEST_METHOD: `PUT` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:04.304927 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53840ee8-ffc3-4e3e-87db-33c2766a6056 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:04.305220 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53840ee8-ffc3-4e3e-87db-33c2766a6056 admin admin] PATH_INFO: `/v3/projects/c64d79783ae4424fa5954e51f833d9ff/users/c42e4cd853f84b07b47be4cd62d79d8b/roles/62f19ea272b447ccaab1576cd9bf4fd9` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:04.319899 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53840ee8-ffc3-4e3e-87db-33c2766a6056 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c64d79783ae4424fa5954e51f833d9ff, user_id=c42e4cd853f84b07b47be4cd62d79d8b, role_id=62f19ea272b447ccaab1576cd9bf4fd9)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:37:04.322596 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53840ee8-ffc3-4e3e-87db-33c2766a6056 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:37:04.399051 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1414/2843] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:37:04 2022] PUT /identity/v3/projects/c64d79783ae4424fa5954e51f833d9ff/users/c42e4cd853f84b07b47be4cd62d79d8b/roles/62f19ea272b447ccaab1576cd9bf4fd9 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:37:04.403149 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cffe76-3b57-4945-a7a1-cc5c48ba046e admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:37:04.413511 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cffe76-3b57-4945-a7a1-cc5c48ba046e admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:37:04.456332 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:37:04.468887 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f8d57a-ac4f-477d-a8ba-57859e2b8ca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:37:04.469492 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f8d57a-ac4f-477d-a8ba-57859e2b8ca4 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:04.469772 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f8d57a-ac4f-477d-a8ba-57859e2b8ca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:04.470046 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f8d57a-ac4f-477d-a8ba-57859e2b8ca4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:04.470721 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f8d57a-ac4f-477d-a8ba-57859e2b8ca4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:37:04.472438 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f8d57a-ac4f-477d-a8ba-57859e2b8ca4 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:37:04.482938 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1430/2844] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:37:04 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:37:04.487098 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53840ee8-ffc3-4e3e-87db-33c2766a6056 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:37:04.498028 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53840ee8-ffc3-4e3e-87db-33c2766a6056 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:37:04.504765 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e85d81-9cb4-4f15-95d9-af291b4df458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:37:04.504765 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e85d81-9cb4-4f15-95d9-af291b4df458 admin admin] REQUEST_METHOD: `PUT` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:04.504765 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e85d81-9cb4-4f15-95d9-af291b4df458 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:04.504765 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e85d81-9cb4-4f15-95d9-af291b4df458 admin admin] PATH_INFO: `/v3/projects/c64d79783ae4424fa5954e51f833d9ff/users/c42e4cd853f84b07b47be4cd62d79d8b/roles/62f19ea272b447ccaab1576cd9bf4fd9` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:04.505807 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e85d81-9cb4-4f15-95d9-af291b4df458 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c64d79783ae4424fa5954e51f833d9ff, user_id=c42e4cd853f84b07b47be4cd62d79d8b, role_id=62f19ea272b447ccaab1576cd9bf4fd9)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:37:04.508473 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e85d81-9cb4-4f15-95d9-af291b4df458 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:37:04.518601 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1415/2845] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:37:04 2022] PUT /identity/v3/projects/c64d79783ae4424fa5954e51f833d9ff/users/c42e4cd853f84b07b47be4cd62d79d8b/roles/62f19ea272b447ccaab1576cd9bf4fd9 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:37:04.528543 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a22f59-7bd3-43ad-b68a-fccaaa92cd3f None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:04.528836 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a22f59-7bd3-43ad-b68a-fccaaa92cd3f None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:04.529123 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a22f59-7bd3-43ad-b68a-fccaaa92cd3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:04.557391 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-40a22f59-7bd3-43ad-b68a-fccaaa92cd3f None None] MFA Rules not processed for user `c42e4cd853f84b07b47be4cd62d79d8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:37:04.558599 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-40a22f59-7bd3-43ad-b68a-fccaaa92cd3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:37:04.628412 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1431/2846] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:37:04 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:37:04.639129 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e85d81-9cb4-4f15-95d9-af291b4df458 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:37:04.640971 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-61e85d81-9cb4-4f15-95d9-af291b4df458 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:37:04.698821 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e85d81-9cb4-4f15-95d9-af291b4df458 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:37:04.702490 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81b2b53-9b2f-47d1-8d67-a000f2297cf2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60193ba9e1c14df3bc9eebcc6f28f2d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:37:04.703091 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81b2b53-9b2f-47d1-8d67-a000f2297cf2 service cinder] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:04.703373 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81b2b53-9b2f-47d1-8d67-a000f2297cf2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:04.703649 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81b2b53-9b2f-47d1-8d67-a000f2297cf2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:04.712940 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81b2b53-9b2f-47d1-8d67-a000f2297cf2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:37:04.714400 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81b2b53-9b2f-47d1-8d67-a000f2297cf2 service cinder] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:37:04.723766 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1416/2847] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:37:04 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:37:05.195750 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a22f59-7bd3-43ad-b68a-fccaaa92cd3f None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:37:05.256326 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a22f59-7bd3-43ad-b68a-fccaaa92cd3f None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:37:05.260800 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ebc679-fe38-4f2a-8210-c10da22d59d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:37:05.261419 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ebc679-fe38-4f2a-8210-c10da22d59d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:05.261701 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ebc679-fe38-4f2a-8210-c10da22d59d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:05.261978 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ebc679-fe38-4f2a-8210-c10da22d59d4 admin admin] PATH_INFO: `/v3/users/201d76cf02a04bb38da134d6bd68d92a` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:05.263122 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ebc679-fe38-4f2a-8210-c10da22d59d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=201d76cf02a04bb38da134d6bd68d92a)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:37:05.265036 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ebc679-fe38-4f2a-8210-c10da22d59d4 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:37:05.411093 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-08ebc679-fe38-4f2a-8210-c10da22d59d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '201d76cf02a04bb38da134d6bd68d92a'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:37:05.461243 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-08ebc679-fe38-4f2a-8210-c10da22d59d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '201d76cf02a04bb38da134d6bd68d92a'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:37:05.461901 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-08ebc679-fe38-4f2a-8210-c10da22d59d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '201d76cf02a04bb38da134d6bd68d92a'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:37:05.469279 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-08ebc679-fe38-4f2a-8210-c10da22d59d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '201d76cf02a04bb38da134d6bd68d92a'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:37:05.474618 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:37:05.481336 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-08ebc679-fe38-4f2a-8210-c10da22d59d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '201d76cf02a04bb38da134d6bd68d92a'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:37:05.487091 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1432/2848] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:37:05 2022] DELETE /identity/v3/users/201d76cf02a04bb38da134d6bd68d92a => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:37:05.492135 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37d4017-d89d-4895-bd67-384815c54b0a None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:05.492435 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37d4017-d89d-4895-bd67-384815c54b0a None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:05.492715 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37d4017-d89d-4895-bd67-384815c54b0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:05.511867 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-f37d4017-d89d-4895-bd67-384815c54b0a None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:37:05.513004 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-f37d4017-d89d-4895-bd67-384815c54b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:37:05.530684 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:37:05.572012 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1417/2849] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:37:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:37:05.581548 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ebc679-fe38-4f2a-8210-c10da22d59d4 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:37:05.582971 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-08ebc679-fe38-4f2a-8210-c10da22d59d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:37:05.639394 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ebc679-fe38-4f2a-8210-c10da22d59d4 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:37:05.643323 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e07f9ef-abea-453b-be0a-86c20f210214 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:37:05.644158 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e07f9ef-abea-453b-be0a-86c20f210214 service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:05.644519 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e07f9ef-abea-453b-be0a-86c20f210214 service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:05.644793 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e07f9ef-abea-453b-be0a-86c20f210214 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:05.654989 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e07f9ef-abea-453b-be0a-86c20f210214 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:37:05.656432 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e07f9ef-abea-453b-be0a-86c20f210214 service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:37:05.666425 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1433/2850] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:37:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:37:06.051483 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37d4017-d89d-4895-bd67-384815c54b0a None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:37:06.053053 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-f37d4017-d89d-4895-bd67-384815c54b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:37:06.065675 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37d4017-d89d-4895-bd67-384815c54b0a None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:37:06.071125 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2f245f-0a7d-453c-ad6f-8f69bb3cafd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:37:06.071788 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2f245f-0a7d-453c-ad6f-8f69bb3cafd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:06.072067 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2f245f-0a7d-453c-ad6f-8f69bb3cafd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:06.072368 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2f245f-0a7d-453c-ad6f-8f69bb3cafd4 admin admin] PATH_INFO: `/v3/projects/6cd7954fca7c45a184567c0a1b091bf9` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:06.074983 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2f245f-0a7d-453c-ad6f-8f69bb3cafd4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6cd7954fca7c45a184567c0a1b091bf9)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:37:06.076297 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2f245f-0a7d-453c-ad6f-8f69bb3cafd4 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:37:06.188050 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-ac2f245f-0a7d-453c-ad6f-8f69bb3cafd4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6cd7954fca7c45a184567c0a1b091bf9', 'request_id': 'req-ac2f245f-0a7d-453c-ad6f-8f69bb3cafd4'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:37:06.249640 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-ac2f245f-0a7d-453c-ad6f-8f69bb3cafd4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6cd7954fca7c45a184567c0a1b091bf9', 'request_id': 'req-ac2f245f-0a7d-453c-ad6f-8f69bb3cafd4'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:37:06.256414 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-ac2f245f-0a7d-453c-ad6f-8f69bb3cafd4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6cd7954fca7c45a184567c0a1b091bf9', 'request_id': 'req-ac2f245f-0a7d-453c-ad6f-8f69bb3cafd4'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:37:06.267290 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-ac2f245f-0a7d-453c-ad6f-8f69bb3cafd4 admin admin] The token cache is being invalidate because project 6cd7954fca7c45a184567c0a1b091bf9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81995) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:37:06.267581 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-ac2f245f-0a7d-453c-ad6f-8f69bb3cafd4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:37:06.268238 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-ac2f245f-0a7d-453c-ad6f-8f69bb3cafd4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:37:06.269747 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1418/2851] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:37:06 2022] DELETE /identity/v3/projects/6cd7954fca7c45a184567c0a1b091bf9 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:37:06.273704 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e07f9ef-abea-453b-be0a-86c20f210214 service neutron] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:37:06.283294 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e07f9ef-abea-453b-be0a-86c20f210214 service neutron] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:37:06.339777 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e60a0e-b3df-47ba-a8e1-e3ad29cdaa50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:37:06.340337 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e60a0e-b3df-47ba-a8e1-e3ad29cdaa50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:06.340619 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e60a0e-b3df-47ba-a8e1-e3ad29cdaa50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:06.340923 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e60a0e-b3df-47ba-a8e1-e3ad29cdaa50 admin admin] PATH_INFO: `/v3/users/c42e4cd853f84b07b47be4cd62d79d8b` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:06.342075 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e60a0e-b3df-47ba-a8e1-e3ad29cdaa50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c42e4cd853f84b07b47be4cd62d79d8b)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:37:06.344037 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e60a0e-b3df-47ba-a8e1-e3ad29cdaa50 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:37:06.531758 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-46e60a0e-b3df-47ba-a8e1-e3ad29cdaa50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c42e4cd853f84b07b47be4cd62d79d8b'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:37:06.599647 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-46e60a0e-b3df-47ba-a8e1-e3ad29cdaa50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c42e4cd853f84b07b47be4cd62d79d8b'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:37:06.600269 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-46e60a0e-b3df-47ba-a8e1-e3ad29cdaa50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c42e4cd853f84b07b47be4cd62d79d8b'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:37:06.607909 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-46e60a0e-b3df-47ba-a8e1-e3ad29cdaa50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c42e4cd853f84b07b47be4cd62d79d8b'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:37:06.620211 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-46e60a0e-b3df-47ba-a8e1-e3ad29cdaa50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c42e4cd853f84b07b47be4cd62d79d8b'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:37:06.626214 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1434/2852] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:37:06 2022] DELETE /identity/v3/users/c42e4cd853f84b07b47be4cd62d79d8b => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:37:07.017197 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2f245f-0a7d-453c-ad6f-8f69bb3cafd4 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:37:07.018726 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-ac2f245f-0a7d-453c-ad6f-8f69bb3cafd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:37:07.033248 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2f245f-0a7d-453c-ad6f-8f69bb3cafd4 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:37:07.091533 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3346738-8e7a-49d6-8176-675d5ad8577a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:37:07.092204 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3346738-8e7a-49d6-8176-675d5ad8577a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:07.092430 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3346738-8e7a-49d6-8176-675d5ad8577a admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:07.092702 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3346738-8e7a-49d6-8176-675d5ad8577a admin admin] PATH_INFO: `/v3/projects/c64d79783ae4424fa5954e51f833d9ff` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:07.093803 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3346738-8e7a-49d6-8176-675d5ad8577a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c64d79783ae4424fa5954e51f833d9ff)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:37:07.095733 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3346738-8e7a-49d6-8176-675d5ad8577a admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:37:07.244712 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-b3346738-8e7a-49d6-8176-675d5ad8577a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c64d79783ae4424fa5954e51f833d9ff', 'request_id': 'req-b3346738-8e7a-49d6-8176-675d5ad8577a'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:37:07.246412 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:37:07.312684 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-b3346738-8e7a-49d6-8176-675d5ad8577a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c64d79783ae4424fa5954e51f833d9ff', 'request_id': 'req-b3346738-8e7a-49d6-8176-675d5ad8577a'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:37:07.318967 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-b3346738-8e7a-49d6-8176-675d5ad8577a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c64d79783ae4424fa5954e51f833d9ff', 'request_id': 'req-b3346738-8e7a-49d6-8176-675d5ad8577a'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:37:07.330412 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-b3346738-8e7a-49d6-8176-675d5ad8577a admin admin] The token cache is being invalidate because project c64d79783ae4424fa5954e51f833d9ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81995) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:37:07.330728 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-b3346738-8e7a-49d6-8176-675d5ad8577a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:37:07.331400 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-b3346738-8e7a-49d6-8176-675d5ad8577a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:37:07.332881 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1419/2853] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:37:07 2022] DELETE /identity/v3/projects/c64d79783ae4424fa5954e51f833d9ff => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:37:07.337920 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae952fe-9d80-482b-ba6f-efbaa803f9fd None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:07.338488 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae952fe-9d80-482b-ba6f-efbaa803f9fd None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:07.338488 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae952fe-9d80-482b-ba6f-efbaa803f9fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:07.360306 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-4ae952fe-9d80-482b-ba6f-efbaa803f9fd None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:37:07.361525 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-4ae952fe-9d80-482b-ba6f-efbaa803f9fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:37:07.365853 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:37:07.420710 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1435/2854] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:37:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:37:07.430443 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d190da-3d20-4691-825f-858a0e494135 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:07.432309 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d190da-3d20-4691-825f-858a0e494135 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:07.432309 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d190da-3d20-4691-825f-858a0e494135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:07.452055 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-52d190da-3d20-4691-825f-858a0e494135 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:37:07.455997 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-52d190da-3d20-4691-825f-858a0e494135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:37:07.461351 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1420/2855] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:37:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:37:07.466150 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae952fe-9d80-482b-ba6f-efbaa803f9fd None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:37:07.476094 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae952fe-9d80-482b-ba6f-efbaa803f9fd None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:37:07.480661 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55059dfd-fdd2-4ea1-a42a-7008d7576300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:37:07.481227 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55059dfd-fdd2-4ea1-a42a-7008d7576300 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:07.481503 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55059dfd-fdd2-4ea1-a42a-7008d7576300 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:07.481771 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55059dfd-fdd2-4ea1-a42a-7008d7576300 admin admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:07.482447 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55059dfd-fdd2-4ea1-a42a-7008d7576300 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:37:07.482748 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55059dfd-fdd2-4ea1-a42a-7008d7576300 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:37:07.484396 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55059dfd-fdd2-4ea1-a42a-7008d7576300 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:37:07.496557 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1436/2856] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Aug 25 03:37:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:37:07.500666 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d190da-3d20-4691-825f-858a0e494135 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:37:07.509941 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d190da-3d20-4691-825f-858a0e494135 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:37:07.514664 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61376dc6-7c8a-41a4-b15e-7d8937535653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:37:07.515264 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61376dc6-7c8a-41a4-b15e-7d8937535653 admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:07.515545 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61376dc6-7c8a-41a4-b15e-7d8937535653 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:07.515817 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61376dc6-7c8a-41a4-b15e-7d8937535653 admin admin] PATH_INFO: `/v3/projects` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:07.516445 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61376dc6-7c8a-41a4-b15e-7d8937535653 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:37:07.518430 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61376dc6-7c8a-41a4-b15e-7d8937535653 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:37:07.589432 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1421/2857] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:37:07 2022] POST /identity/v3/projects => generated 373 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:37:07.593614 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55059dfd-fdd2-4ea1-a42a-7008d7576300 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:37:07.604198 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55059dfd-fdd2-4ea1-a42a-7008d7576300 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:37:07.665600 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2db5e9-f980-4e7c-9f57-97da8013581b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:37:07.666219 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2db5e9-f980-4e7c-9f57-97da8013581b admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:07.666539 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2db5e9-f980-4e7c-9f57-97da8013581b admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:07.666815 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2db5e9-f980-4e7c-9f57-97da8013581b admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:07.667448 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2db5e9-f980-4e7c-9f57-97da8013581b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:37:07.669415 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2db5e9-f980-4e7c-9f57-97da8013581b admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:37:07.741848 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1437/2858] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:37:07 2022] POST /identity/v3/users => generated 287 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:37:07.746116 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61376dc6-7c8a-41a4-b15e-7d8937535653 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:37:07.756532 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61376dc6-7c8a-41a4-b15e-7d8937535653 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:37:07.761111 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34269454-a05f-47cf-a290-1816311cce36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:37:07.761707 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34269454-a05f-47cf-a290-1816311cce36 admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:07.761989 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34269454-a05f-47cf-a290-1816311cce36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:07.762259 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34269454-a05f-47cf-a290-1816311cce36 admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:07.762930 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34269454-a05f-47cf-a290-1816311cce36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:37:07.764531 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34269454-a05f-47cf-a290-1816311cce36 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:37:07.776703 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1422/2859] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:37:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:37:07.780841 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2db5e9-f980-4e7c-9f57-97da8013581b admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:37:07.791452 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2db5e9-f980-4e7c-9f57-97da8013581b admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:37:07.796015 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d3a048-9853-4012-919e-f804f2e8f3dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:37:07.796605 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d3a048-9853-4012-919e-f804f2e8f3dc admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:07.796883 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d3a048-9853-4012-919e-f804f2e8f3dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:07.797175 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d3a048-9853-4012-919e-f804f2e8f3dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:07.797813 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d3a048-9853-4012-919e-f804f2e8f3dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:37:07.799432 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d3a048-9853-4012-919e-f804f2e8f3dc admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:37:07.810090 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1438/2860] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:37:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:37:07.814431 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34269454-a05f-47cf-a290-1816311cce36 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:37:07.824599 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34269454-a05f-47cf-a290-1816311cce36 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:37:07.829413 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762a143b-97ba-48c2-9769-bc9f1dd62008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:37:07.830105 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762a143b-97ba-48c2-9769-bc9f1dd62008 admin admin] REQUEST_METHOD: `PUT` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:07.830485 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762a143b-97ba-48c2-9769-bc9f1dd62008 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:07.830765 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762a143b-97ba-48c2-9769-bc9f1dd62008 admin admin] PATH_INFO: `/v3/projects/4e2b870f784c4482aa0c343849f7963b/users/a906121e431e432e8ccddca219b25134/roles/3255f77192524a16bc405173106871fc` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:07.845451 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762a143b-97ba-48c2-9769-bc9f1dd62008 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e2b870f784c4482aa0c343849f7963b, user_id=a906121e431e432e8ccddca219b25134, role_id=3255f77192524a16bc405173106871fc)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:37:07.848190 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762a143b-97ba-48c2-9769-bc9f1dd62008 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:37:07.913539 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1423/2861] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:37:07 2022] PUT /identity/v3/projects/4e2b870f784c4482aa0c343849f7963b/users/a906121e431e432e8ccddca219b25134/roles/3255f77192524a16bc405173106871fc => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:37:07.918773 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33c338a-acfb-483e-8067-6d4a66de777d None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:07.919181 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33c338a-acfb-483e-8067-6d4a66de777d None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:07.919463 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33c338a-acfb-483e-8067-6d4a66de777d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:07.938487 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-c33c338a-acfb-483e-8067-6d4a66de777d None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:37:07.939691 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-c33c338a-acfb-483e-8067-6d4a66de777d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:37:08.000106 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1439/2862] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:37:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:37:08.010338 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762a143b-97ba-48c2-9769-bc9f1dd62008 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:37:08.011972 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-762a143b-97ba-48c2-9769-bc9f1dd62008 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:37:08.070071 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762a143b-97ba-48c2-9769-bc9f1dd62008 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:37:08.073839 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a89bd9-555c-4ab9-b9c6-0bc9b22019d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:37:08.074482 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a89bd9-555c-4ab9-b9c6-0bc9b22019d2 service neutron] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:08.074777 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a89bd9-555c-4ab9-b9c6-0bc9b22019d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:08.075035 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a89bd9-555c-4ab9-b9c6-0bc9b22019d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:08.105271 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a89bd9-555c-4ab9-b9c6-0bc9b22019d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:37:08.106763 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a89bd9-555c-4ab9-b9c6-0bc9b22019d2 service neutron] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:37:08.116582 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1424/2863] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:37:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:37:09.324991 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6461a28-f253-4cbd-a774-38b74da686ad None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:09.325522 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6461a28-f253-4cbd-a774-38b74da686ad None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:09.325628 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6461a28-f253-4cbd-a774-38b74da686ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:09.351372 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:37:09.365087 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-d6461a28-f253-4cbd-a774-38b74da686ad None None] MFA Rules not processed for user `a906121e431e432e8ccddca219b25134`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:37:09.366392 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-d6461a28-f253-4cbd-a774-38b74da686ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:37:09.434086 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1440/2864] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:37:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:37:09.438724 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a89bd9-555c-4ab9-b9c6-0bc9b22019d2 service neutron] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:37:09.470546 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a89bd9-555c-4ab9-b9c6-0bc9b22019d2 service neutron] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:37:09.475246 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bddfbc-2fbb-4714-a74e-3013df24521c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:37:09.475867 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bddfbc-2fbb-4714-a74e-3013df24521c admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:09.476149 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bddfbc-2fbb-4714-a74e-3013df24521c admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:09.476422 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bddfbc-2fbb-4714-a74e-3013df24521c admin admin] PATH_INFO: `/v3/projects` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:09.477047 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bddfbc-2fbb-4714-a74e-3013df24521c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:37:09.478984 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bddfbc-2fbb-4714-a74e-3013df24521c admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:37:09.572941 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1425/2865] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:37:09 2022] POST /identity/v3/projects => generated 373 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:37:09.576988 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6461a28-f253-4cbd-a774-38b74da686ad None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:37:09.588622 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6461a28-f253-4cbd-a774-38b74da686ad None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:37:09.650383 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe57405-2269-4958-b59b-28a02efcc208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:37:09.651002 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe57405-2269-4958-b59b-28a02efcc208 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:09.651288 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe57405-2269-4958-b59b-28a02efcc208 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:09.651565 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe57405-2269-4958-b59b-28a02efcc208 admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:09.652201 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe57405-2269-4958-b59b-28a02efcc208 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:37:09.654125 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe57405-2269-4958-b59b-28a02efcc208 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:37:09.724023 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1441/2866] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:37:09 2022] POST /identity/v3/users => generated 293 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:37:09.727978 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bddfbc-2fbb-4714-a74e-3013df24521c admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:37:09.734737 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:37:09.738376 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bddfbc-2fbb-4714-a74e-3013df24521c admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:37:09.743102 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f43d52-0e14-4168-99b6-eb0e36b448a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:37:09.743701 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f43d52-0e14-4168-99b6-eb0e36b448a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:09.743996 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f43d52-0e14-4168-99b6-eb0e36b448a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:09.744251 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f43d52-0e14-4168-99b6-eb0e36b448a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:09.744891 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f43d52-0e14-4168-99b6-eb0e36b448a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:37:09.746570 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f43d52-0e14-4168-99b6-eb0e36b448a4 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:37:09.756553 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1426/2867] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:37:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:37:09.761460 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe57405-2269-4958-b59b-28a02efcc208 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:37:09.771224 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe57405-2269-4958-b59b-28a02efcc208 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:37:09.776744 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f81c57-35bc-46f3-a07d-365ca908a540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:37:09.777397 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f81c57-35bc-46f3-a07d-365ca908a540 admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:09.777705 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f81c57-35bc-46f3-a07d-365ca908a540 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:09.777982 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f81c57-35bc-46f3-a07d-365ca908a540 admin admin] PATH_INFO: `/v3/projects/bacb72654758447593473923a343bef3/users/3fb9cf7e920a44c7bb36aaa635309e0b/roles/62f19ea272b447ccaab1576cd9bf4fd9` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:09.795218 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f81c57-35bc-46f3-a07d-365ca908a540 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bacb72654758447593473923a343bef3, user_id=3fb9cf7e920a44c7bb36aaa635309e0b, role_id=62f19ea272b447ccaab1576cd9bf4fd9)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:37:09.797993 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f81c57-35bc-46f3-a07d-365ca908a540 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:37:09.861139 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1442/2868] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:37:09 2022] PUT /identity/v3/projects/bacb72654758447593473923a343bef3/users/3fb9cf7e920a44c7bb36aaa635309e0b/roles/62f19ea272b447ccaab1576cd9bf4fd9 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:37:09.865244 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f43d52-0e14-4168-99b6-eb0e36b448a4 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:37:09.875970 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f43d52-0e14-4168-99b6-eb0e36b448a4 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:37:09.932376 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c59322f-2d2e-43cd-9525-7d5bc90e96c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:37:09.932992 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c59322f-2d2e-43cd-9525-7d5bc90e96c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:09.933275 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c59322f-2d2e-43cd-9525-7d5bc90e96c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:09.933555 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c59322f-2d2e-43cd-9525-7d5bc90e96c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:09.934204 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c59322f-2d2e-43cd-9525-7d5bc90e96c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:37:09.935846 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c59322f-2d2e-43cd-9525-7d5bc90e96c1 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:37:09.948296 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1427/2869] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:37:09 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:37:09.952462 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f81c57-35bc-46f3-a07d-365ca908a540 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:37:09.962650 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f81c57-35bc-46f3-a07d-365ca908a540 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:37:09.967214 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f184b6-5739-4b35-88e9-1c479dfa0711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:37:09.967859 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f184b6-5739-4b35-88e9-1c479dfa0711 admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:09.968136 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f184b6-5739-4b35-88e9-1c479dfa0711 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:09.968408 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f184b6-5739-4b35-88e9-1c479dfa0711 admin admin] PATH_INFO: `/v3/projects/bacb72654758447593473923a343bef3/users/3fb9cf7e920a44c7bb36aaa635309e0b/roles/62f19ea272b447ccaab1576cd9bf4fd9` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:09.970417 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f184b6-5739-4b35-88e9-1c479dfa0711 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bacb72654758447593473923a343bef3, user_id=3fb9cf7e920a44c7bb36aaa635309e0b, role_id=62f19ea272b447ccaab1576cd9bf4fd9)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:37:09.973021 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f184b6-5739-4b35-88e9-1c479dfa0711 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:37:09.984240 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1443/2870] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:37:09 2022] PUT /identity/v3/projects/bacb72654758447593473923a343bef3/users/3fb9cf7e920a44c7bb36aaa635309e0b/roles/62f19ea272b447ccaab1576cd9bf4fd9 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:37:11.519970 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7ef3d1-b25f-4100-8fd2-520ad1372a49 None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:11.520496 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7ef3d1-b25f-4100-8fd2-520ad1372a49 None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:11.520991 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7ef3d1-b25f-4100-8fd2-520ad1372a49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:11.559602 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-2d7ef3d1-b25f-4100-8fd2-520ad1372a49 None None] MFA Rules not processed for user `3fb9cf7e920a44c7bb36aaa635309e0b`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:37:11.561441 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-2d7ef3d1-b25f-4100-8fd2-520ad1372a49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:37:11.651297 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1428/2871] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:37:11 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:37:11.663208 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f184b6-5739-4b35-88e9-1c479dfa0711 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:37:11.665371 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-60f184b6-5739-4b35-88e9-1c479dfa0711 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:37:11.716284 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:37:11.740136 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f184b6-5739-4b35-88e9-1c479dfa0711 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:37:11.744566 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb780b2-16f7-4f76-85df-6a135421885c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60193ba9e1c14df3bc9eebcc6f28f2d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:37:11.745185 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb780b2-16f7-4f76-85df-6a135421885c service cinder] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:11.745467 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb780b2-16f7-4f76-85df-6a135421885c service cinder] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:11.745721 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb780b2-16f7-4f76-85df-6a135421885c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:11.805894 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb780b2-16f7-4f76-85df-6a135421885c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:37:11.807672 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb780b2-16f7-4f76-85df-6a135421885c service cinder] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:37:11.816648 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1444/2872] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:37:11 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:37:13.634435 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7ef3d1-b25f-4100-8fd2-520ad1372a49 None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:37:13.635890 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-2d7ef3d1-b25f-4100-8fd2-520ad1372a49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:37:13.648658 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7ef3d1-b25f-4100-8fd2-520ad1372a49 None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:37:13.652034 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873acf70-a95a-48ac-bfa0-ad4153aaf2ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60193ba9e1c14df3bc9eebcc6f28f2d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:37:13.652635 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873acf70-a95a-48ac-bfa0-ad4153aaf2ff service cinder] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:13.652940 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873acf70-a95a-48ac-bfa0-ad4153aaf2ff service cinder] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:13.653211 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873acf70-a95a-48ac-bfa0-ad4153aaf2ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:13.662081 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873acf70-a95a-48ac-bfa0-ad4153aaf2ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:37:13.664205 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873acf70-a95a-48ac-bfa0-ad4153aaf2ff service cinder] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:37:13.673495 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1429/2873] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:37:13 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:37:21.812714 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=35 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:37:23.669247 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=72 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:37:30.542216 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb780b2-16f7-4f76-85df-6a135421885c service cinder] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:37:30.543669 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-1eb780b2-16f7-4f76-85df-6a135421885c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:37:30.639039 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb780b2-16f7-4f76-85df-6a135421885c service cinder] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:37:30.644343 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f767174c-983e-49f0-8a6c-910041f90041 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b692edd3e8c41289cc42e1afb4a6dd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:37:30.647670 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f767174c-983e-49f0-8a6c-910041f90041 service placement] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:30.647670 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f767174c-983e-49f0-8a6c-910041f90041 service placement] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:30.647670 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f767174c-983e-49f0-8a6c-910041f90041 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:30.647670 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-f767174c-983e-49f0-8a6c-910041f90041 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:37:30.658829 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f767174c-983e-49f0-8a6c-910041f90041 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:37:30.660403 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f767174c-983e-49f0-8a6c-910041f90041 service placement] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:37:30.670999 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1445/2874] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:37:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:37:31.179824 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873acf70-a95a-48ac-bfa0-ad4153aaf2ff service cinder] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:37:31.184260 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873acf70-a95a-48ac-bfa0-ad4153aaf2ff service cinder] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:37:31.189740 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcac8777-8d75-47d4-bdfd-2866506208d2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b692edd3e8c41289cc42e1afb4a6dd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:37:31.190417 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcac8777-8d75-47d4-bdfd-2866506208d2 service placement] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:31.190704 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcac8777-8d75-47d4-bdfd-2866506208d2 service placement] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:31.190980 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcac8777-8d75-47d4-bdfd-2866506208d2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:31.192850 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-bcac8777-8d75-47d4-bdfd-2866506208d2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:37:31.205653 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcac8777-8d75-47d4-bdfd-2866506208d2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:37:31.207265 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcac8777-8d75-47d4-bdfd-2866506208d2 service placement] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:37:31.218336 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1430/2875] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:37:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:37:31.295841 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f767174c-983e-49f0-8a6c-910041f90041 service placement] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:37:31.359305 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f767174c-983e-49f0-8a6c-910041f90041 service placement] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:37:31.363840 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0255be-7854-4855-b0a3-e6bb59bf89ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:37:31.364467 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0255be-7854-4855-b0a3-e6bb59bf89ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:31.364749 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0255be-7854-4855-b0a3-e6bb59bf89ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:31.365024 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0255be-7854-4855-b0a3-e6bb59bf89ff admin admin] PATH_INFO: `/v3/users/a906121e431e432e8ccddca219b25134` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:31.366143 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0255be-7854-4855-b0a3-e6bb59bf89ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=a906121e431e432e8ccddca219b25134)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:37:31.368095 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0255be-7854-4855-b0a3-e6bb59bf89ff admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:37:31.503422 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-ad0255be-7854-4855-b0a3-e6bb59bf89ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a906121e431e432e8ccddca219b25134'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:37:31.546564 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-ad0255be-7854-4855-b0a3-e6bb59bf89ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a906121e431e432e8ccddca219b25134'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:37:31.547167 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-ad0255be-7854-4855-b0a3-e6bb59bf89ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a906121e431e432e8ccddca219b25134'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:37:31.554090 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-ad0255be-7854-4855-b0a3-e6bb59bf89ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a906121e431e432e8ccddca219b25134'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:37:31.562698 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:37:31.567062 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-ad0255be-7854-4855-b0a3-e6bb59bf89ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a906121e431e432e8ccddca219b25134'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:37:31.573530 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1446/2876] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:37:31 2022] DELETE /identity/v3/users/a906121e431e432e8ccddca219b25134 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:37:31.829553 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcac8777-8d75-47d4-bdfd-2866506208d2 service placement] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:37:31.842273 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcac8777-8d75-47d4-bdfd-2866506208d2 service placement] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:37:31.900107 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042a920a-b934-49bf-bd45-8dfb1da4fac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:37:31.900737 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042a920a-b934-49bf-bd45-8dfb1da4fac7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:31.901014 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042a920a-b934-49bf-bd45-8dfb1da4fac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:31.901294 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042a920a-b934-49bf-bd45-8dfb1da4fac7 admin admin] PATH_INFO: `/v3/projects/4e2b870f784c4482aa0c343849f7963b` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:31.902446 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042a920a-b934-49bf-bd45-8dfb1da4fac7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e2b870f784c4482aa0c343849f7963b)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:37:31.904352 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042a920a-b934-49bf-bd45-8dfb1da4fac7 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:37:32.004713 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-042a920a-b934-49bf-bd45-8dfb1da4fac7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e2b870f784c4482aa0c343849f7963b', 'request_id': 'req-042a920a-b934-49bf-bd45-8dfb1da4fac7'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:37:32.066175 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-042a920a-b934-49bf-bd45-8dfb1da4fac7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e2b870f784c4482aa0c343849f7963b', 'request_id': 'req-042a920a-b934-49bf-bd45-8dfb1da4fac7'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:37:32.071910 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-042a920a-b934-49bf-bd45-8dfb1da4fac7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e2b870f784c4482aa0c343849f7963b', 'request_id': 'req-042a920a-b934-49bf-bd45-8dfb1da4fac7'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:37:32.082851 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-042a920a-b934-49bf-bd45-8dfb1da4fac7 admin admin] The token cache is being invalidate because project 4e2b870f784c4482aa0c343849f7963b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81995) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:37:32.084400 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-042a920a-b934-49bf-bd45-8dfb1da4fac7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:37:32.084400 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-042a920a-b934-49bf-bd45-8dfb1da4fac7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:37:32.085292 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1431/2877] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:37:31 2022] DELETE /identity/v3/projects/4e2b870f784c4482aa0c343849f7963b => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:37:32.089486 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0255be-7854-4855-b0a3-e6bb59bf89ff admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:37:32.090907 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-ad0255be-7854-4855-b0a3-e6bb59bf89ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:37:32.103757 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0255be-7854-4855-b0a3-e6bb59bf89ff admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:37:32.159460 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f84bc9-c2ef-4543-9664-8aed2f4919d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:37:32.160094 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f84bc9-c2ef-4543-9664-8aed2f4919d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:32.160376 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f84bc9-c2ef-4543-9664-8aed2f4919d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:32.160652 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f84bc9-c2ef-4543-9664-8aed2f4919d4 admin admin] PATH_INFO: `/v3/users/3fb9cf7e920a44c7bb36aaa635309e0b` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:32.161847 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f84bc9-c2ef-4543-9664-8aed2f4919d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3fb9cf7e920a44c7bb36aaa635309e0b)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:37:32.163801 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f84bc9-c2ef-4543-9664-8aed2f4919d4 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:37:32.371953 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-b6f84bc9-c2ef-4543-9664-8aed2f4919d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3fb9cf7e920a44c7bb36aaa635309e0b'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:37:32.432648 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-b6f84bc9-c2ef-4543-9664-8aed2f4919d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3fb9cf7e920a44c7bb36aaa635309e0b'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:37:32.433279 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-b6f84bc9-c2ef-4543-9664-8aed2f4919d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3fb9cf7e920a44c7bb36aaa635309e0b'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:37:32.440913 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-b6f84bc9-c2ef-4543-9664-8aed2f4919d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3fb9cf7e920a44c7bb36aaa635309e0b'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:37:32.452665 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-b6f84bc9-c2ef-4543-9664-8aed2f4919d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3fb9cf7e920a44c7bb36aaa635309e0b'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:37:32.458552 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1447/2878] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:37:32 2022] DELETE /identity/v3/users/3fb9cf7e920a44c7bb36aaa635309e0b => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:37:32.686246 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042a920a-b934-49bf-bd45-8dfb1da4fac7 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:37:32.687746 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-042a920a-b934-49bf-bd45-8dfb1da4fac7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:37:32.703091 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042a920a-b934-49bf-bd45-8dfb1da4fac7 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:37:32.765960 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6adce2-3a25-4bcf-859d-5ec2b3552380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:37:32.766631 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6adce2-3a25-4bcf-859d-5ec2b3552380 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:32.766916 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6adce2-3a25-4bcf-859d-5ec2b3552380 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:32.767190 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6adce2-3a25-4bcf-859d-5ec2b3552380 admin admin] PATH_INFO: `/v3/projects/bacb72654758447593473923a343bef3` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:32.768304 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6adce2-3a25-4bcf-859d-5ec2b3552380 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bacb72654758447593473923a343bef3)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:37:32.770235 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6adce2-3a25-4bcf-859d-5ec2b3552380 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:37:32.792147 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:37:32.866799 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-dd6adce2-3a25-4bcf-859d-5ec2b3552380 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bacb72654758447593473923a343bef3', 'request_id': 'req-dd6adce2-3a25-4bcf-859d-5ec2b3552380'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:37:32.918226 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-dd6adce2-3a25-4bcf-859d-5ec2b3552380 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bacb72654758447593473923a343bef3', 'request_id': 'req-dd6adce2-3a25-4bcf-859d-5ec2b3552380'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:37:32.923889 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-dd6adce2-3a25-4bcf-859d-5ec2b3552380 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bacb72654758447593473923a343bef3', 'request_id': 'req-dd6adce2-3a25-4bcf-859d-5ec2b3552380'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:37:32.935332 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-dd6adce2-3a25-4bcf-859d-5ec2b3552380 admin admin] The token cache is being invalidate because project bacb72654758447593473923a343bef3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81995) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:37:32.935625 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-dd6adce2-3a25-4bcf-859d-5ec2b3552380 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:37:32.936253 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-dd6adce2-3a25-4bcf-859d-5ec2b3552380 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:37:32.937728 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1432/2879] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:37:32 2022] DELETE /identity/v3/projects/bacb72654758447593473923a343bef3 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:37:32.942740 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d8055e-054e-4f66-99f0-33a349f1c0b7 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:32.943282 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d8055e-054e-4f66-99f0-33a349f1c0b7 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:32.943469 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d8055e-054e-4f66-99f0-33a349f1c0b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:33.001571 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-e9d8055e-054e-4f66-99f0-33a349f1c0b7 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:37:33.002822 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-e9d8055e-054e-4f66-99f0-33a349f1c0b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:37:33.035055 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:37:33.061280 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1448/2880] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:37:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:37:33.069501 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fa8fca-3b63-435f-9d1a-a9c8b9dac36d None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:33.069789 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fa8fca-3b63-435f-9d1a-a9c8b9dac36d None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:33.075207 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fa8fca-3b63-435f-9d1a-a9c8b9dac36d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:33.089936 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-96fa8fca-3b63-435f-9d1a-a9c8b9dac36d None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:37:33.092442 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-96fa8fca-3b63-435f-9d1a-a9c8b9dac36d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:37:33.099020 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1433/2881] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:37:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:37:33.103343 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d8055e-054e-4f66-99f0-33a349f1c0b7 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:37:33.112777 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d8055e-054e-4f66-99f0-33a349f1c0b7 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:37:33.117330 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38efda5c-e137-4072-ac0c-e4000b7ba722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:37:33.117928 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38efda5c-e137-4072-ac0c-e4000b7ba722 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:33.118211 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38efda5c-e137-4072-ac0c-e4000b7ba722 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:33.118516 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38efda5c-e137-4072-ac0c-e4000b7ba722 admin admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:33.119161 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38efda5c-e137-4072-ac0c-e4000b7ba722 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:37:33.119468 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38efda5c-e137-4072-ac0c-e4000b7ba722 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:37:33.121570 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38efda5c-e137-4072-ac0c-e4000b7ba722 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:37:33.132790 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1449/2882] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Aug 25 03:37:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:37:33.137580 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fa8fca-3b63-435f-9d1a-a9c8b9dac36d None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:37:33.146647 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fa8fca-3b63-435f-9d1a-a9c8b9dac36d None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:37:33.151199 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e99c196-1597-49aa-a454-9427c6ab579e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:37:33.151787 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e99c196-1597-49aa-a454-9427c6ab579e admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:33.152061 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e99c196-1597-49aa-a454-9427c6ab579e admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:33.152329 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e99c196-1597-49aa-a454-9427c6ab579e admin admin] PATH_INFO: `/v3/projects` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:33.152940 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e99c196-1597-49aa-a454-9427c6ab579e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:37:33.154849 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e99c196-1597-49aa-a454-9427c6ab579e admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:37:33.234473 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1434/2883] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:37:33 2022] POST /identity/v3/projects => generated 381 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:37:33.238533 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38efda5c-e137-4072-ac0c-e4000b7ba722 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:37:33.249050 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38efda5c-e137-4072-ac0c-e4000b7ba722 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:37:33.311334 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe5b750-51a6-47d5-9807-ed58057c8388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:37:33.311946 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe5b750-51a6-47d5-9807-ed58057c8388 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:33.312230 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe5b750-51a6-47d5-9807-ed58057c8388 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:33.312508 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe5b750-51a6-47d5-9807-ed58057c8388 admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:33.313136 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe5b750-51a6-47d5-9807-ed58057c8388 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:37:33.315094 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe5b750-51a6-47d5-9807-ed58057c8388 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:37:33.392573 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1450/2884] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:37:33 2022] POST /identity/v3/users => generated 291 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:37:33.396856 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e99c196-1597-49aa-a454-9427c6ab579e admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:37:33.407296 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e99c196-1597-49aa-a454-9427c6ab579e admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:37:33.411869 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ddae2e-77d6-4b10-b03e-815a4d5a89bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:37:33.412449 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ddae2e-77d6-4b10-b03e-815a4d5a89bd admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:33.412731 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ddae2e-77d6-4b10-b03e-815a4d5a89bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:33.413005 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ddae2e-77d6-4b10-b03e-815a4d5a89bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:33.413643 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ddae2e-77d6-4b10-b03e-815a4d5a89bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:37:33.415318 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ddae2e-77d6-4b10-b03e-815a4d5a89bd admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:37:33.425713 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1435/2885] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:37:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:37:33.429926 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe5b750-51a6-47d5-9807-ed58057c8388 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:37:33.440812 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe5b750-51a6-47d5-9807-ed58057c8388 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:37:33.445431 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1368a166-48d6-4d19-a622-bfbccfee46d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:37:33.446021 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1368a166-48d6-4d19-a622-bfbccfee46d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:33.446324 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1368a166-48d6-4d19-a622-bfbccfee46d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:33.446604 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1368a166-48d6-4d19-a622-bfbccfee46d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:33.447245 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1368a166-48d6-4d19-a622-bfbccfee46d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:37:33.448834 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1368a166-48d6-4d19-a622-bfbccfee46d3 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:37:33.459207 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1451/2886] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:37:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:37:33.463177 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ddae2e-77d6-4b10-b03e-815a4d5a89bd admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:37:33.473050 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ddae2e-77d6-4b10-b03e-815a4d5a89bd admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:37:33.477807 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2321ec12-ba7d-4f09-a9a1-252e5286ec88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:37:33.478488 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2321ec12-ba7d-4f09-a9a1-252e5286ec88 admin admin] REQUEST_METHOD: `PUT` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:33.478775 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2321ec12-ba7d-4f09-a9a1-252e5286ec88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:33.479054 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2321ec12-ba7d-4f09-a9a1-252e5286ec88 admin admin] PATH_INFO: `/v3/projects/b9124fd8b4394c3b90e6b871eed8380e/users/15291fd8a5ca4e7db90d94b5ed2f529a/roles/3255f77192524a16bc405173106871fc` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:33.493791 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2321ec12-ba7d-4f09-a9a1-252e5286ec88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9124fd8b4394c3b90e6b871eed8380e, user_id=15291fd8a5ca4e7db90d94b5ed2f529a, role_id=3255f77192524a16bc405173106871fc)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:37:33.497049 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2321ec12-ba7d-4f09-a9a1-252e5286ec88 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:37:33.558385 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1436/2887] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:37:33 2022] PUT /identity/v3/projects/b9124fd8b4394c3b90e6b871eed8380e/users/15291fd8a5ca4e7db90d94b5ed2f529a/roles/3255f77192524a16bc405173106871fc => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:37:33.568496 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5356753f-ec41-4dd0-8b7e-ba338a552719 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:33.568782 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5356753f-ec41-4dd0-8b7e-ba338a552719 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:33.569081 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5356753f-ec41-4dd0-8b7e-ba338a552719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:33.596699 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-5356753f-ec41-4dd0-8b7e-ba338a552719 None None] MFA Rules not processed for user `15291fd8a5ca4e7db90d94b5ed2f529a`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:37:33.597886 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-5356753f-ec41-4dd0-8b7e-ba338a552719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:37:33.662234 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1452/2888] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:37:33 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:37:33.667213 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2321ec12-ba7d-4f09-a9a1-252e5286ec88 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:37:33.727321 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2321ec12-ba7d-4f09-a9a1-252e5286ec88 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:37:33.732535 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55896d72-58a9-43ff-b3ca-c782e49c1c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:37:33.733140 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55896d72-58a9-43ff-b3ca-c782e49c1c2b admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:33.733423 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55896d72-58a9-43ff-b3ca-c782e49c1c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:33.733702 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55896d72-58a9-43ff-b3ca-c782e49c1c2b admin admin] PATH_INFO: `/v3/projects` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:33.734364 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55896d72-58a9-43ff-b3ca-c782e49c1c2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:37:33.736279 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55896d72-58a9-43ff-b3ca-c782e49c1c2b admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:37:33.804730 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1437/2889] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:37:33 2022] POST /identity/v3/projects => generated 381 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:37:33.809078 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5356753f-ec41-4dd0-8b7e-ba338a552719 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:37:33.819758 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5356753f-ec41-4dd0-8b7e-ba338a552719 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:37:33.849308 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:37:33.875288 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a6e2ba-a652-4347-aec4-03d154631747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:37:33.875913 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a6e2ba-a652-4347-aec4-03d154631747 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:33.876190 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a6e2ba-a652-4347-aec4-03d154631747 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:33.876460 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a6e2ba-a652-4347-aec4-03d154631747 admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:33.877096 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a6e2ba-a652-4347-aec4-03d154631747 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:37:33.879042 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a6e2ba-a652-4347-aec4-03d154631747 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:37:33.940793 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1453/2890] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:37:33 2022] POST /identity/v3/users => generated 291 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:37:33.945007 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55896d72-58a9-43ff-b3ca-c782e49c1c2b admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:37:33.955329 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55896d72-58a9-43ff-b3ca-c782e49c1c2b admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:37:33.960062 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72e5cb3-47d0-479e-a9d2-4954ef1a1b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:37:33.960659 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72e5cb3-47d0-479e-a9d2-4954ef1a1b29 admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:33.960940 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72e5cb3-47d0-479e-a9d2-4954ef1a1b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:33.961215 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72e5cb3-47d0-479e-a9d2-4954ef1a1b29 admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:33.961859 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72e5cb3-47d0-479e-a9d2-4954ef1a1b29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:37:33.963542 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72e5cb3-47d0-479e-a9d2-4954ef1a1b29 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:37:33.974468 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1438/2891] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:37:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:37:33.978312 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a6e2ba-a652-4347-aec4-03d154631747 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:37:33.988822 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a6e2ba-a652-4347-aec4-03d154631747 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:37:33.993353 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8cb265-32ec-43a6-9150-2edb44898118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:37:33.993937 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8cb265-32ec-43a6-9150-2edb44898118 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:33.994221 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8cb265-32ec-43a6-9150-2edb44898118 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:33.994521 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8cb265-32ec-43a6-9150-2edb44898118 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:33.995154 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8cb265-32ec-43a6-9150-2edb44898118 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:37:33.996722 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8cb265-32ec-43a6-9150-2edb44898118 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:37:34.006447 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1454/2892] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:37:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:37:34.010503 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72e5cb3-47d0-479e-a9d2-4954ef1a1b29 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:37:34.020399 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72e5cb3-47d0-479e-a9d2-4954ef1a1b29 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:37:34.024987 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe28549-d752-439c-8e43-e13241b0cbc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:37:34.025623 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe28549-d752-439c-8e43-e13241b0cbc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:34.025907 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe28549-d752-439c-8e43-e13241b0cbc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:34.026710 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe28549-d752-439c-8e43-e13241b0cbc3 admin admin] PATH_INFO: `/v3/projects/05de7488ec9c413da49551fcf3b09bc3/users/c239889079434d13bb1d47f9ad240a1a/roles/3255f77192524a16bc405173106871fc` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:34.040932 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe28549-d752-439c-8e43-e13241b0cbc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05de7488ec9c413da49551fcf3b09bc3, user_id=c239889079434d13bb1d47f9ad240a1a, role_id=3255f77192524a16bc405173106871fc)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:37:34.044212 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe28549-d752-439c-8e43-e13241b0cbc3 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:37:34.046409 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:37:34.095528 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1439/2893] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:37:34 2022] PUT /identity/v3/projects/05de7488ec9c413da49551fcf3b09bc3/users/c239889079434d13bb1d47f9ad240a1a/roles/3255f77192524a16bc405173106871fc => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:37:34.107239 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fdf01c-2214-4593-9de7-469be0d92955 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:34.107531 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fdf01c-2214-4593-9de7-469be0d92955 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:34.107811 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fdf01c-2214-4593-9de7-469be0d92955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:34.135065 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-a0fdf01c-2214-4593-9de7-469be0d92955 None None] MFA Rules not processed for user `c239889079434d13bb1d47f9ad240a1a`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:37:34.136823 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-a0fdf01c-2214-4593-9de7-469be0d92955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:37:34.200524 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1455/2894] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:37:34 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:37:34.205030 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe28549-d752-439c-8e43-e13241b0cbc3 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:37:34.266497 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe28549-d752-439c-8e43-e13241b0cbc3 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:37:34.271366 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddea2dc0-ff26-4397-905e-ac5449fbe67e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:37:34.271974 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddea2dc0-ff26-4397-905e-ac5449fbe67e admin admin] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:34.272252 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddea2dc0-ff26-4397-905e-ac5449fbe67e admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:34.272524 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddea2dc0-ff26-4397-905e-ac5449fbe67e admin admin] PATH_INFO: `/v3/projects` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:34.273148 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddea2dc0-ff26-4397-905e-ac5449fbe67e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:37:34.275095 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddea2dc0-ff26-4397-905e-ac5449fbe67e admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:37:34.360005 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1440/2895] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Aug 25 03:37:34 2022] POST /identity/v3/projects => generated 383 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:37:34.364110 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fdf01c-2214-4593-9de7-469be0d92955 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:37:34.374721 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fdf01c-2214-4593-9de7-469be0d92955 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:37:34.429618 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79db790e-4775-4db3-9d38-3eee3503bc43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:37:34.430225 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79db790e-4775-4db3-9d38-3eee3503bc43 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:34.430656 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79db790e-4775-4db3-9d38-3eee3503bc43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:34.430958 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79db790e-4775-4db3-9d38-3eee3503bc43 admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:34.431855 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79db790e-4775-4db3-9d38-3eee3503bc43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:37:34.433771 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79db790e-4775-4db3-9d38-3eee3503bc43 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:37:34.513122 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1456/2896] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Aug 25 03:37:34 2022] POST /identity/v3/users => generated 298 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:37:34.517143 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddea2dc0-ff26-4397-905e-ac5449fbe67e admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:37:34.527478 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddea2dc0-ff26-4397-905e-ac5449fbe67e admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:37:34.532475 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc738260-547f-4de3-908c-5463ebc667bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:37:34.532847 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc738260-547f-4de3-908c-5463ebc667bd admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:34.533140 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc738260-547f-4de3-908c-5463ebc667bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:34.533414 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc738260-547f-4de3-908c-5463ebc667bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:34.534057 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc738260-547f-4de3-908c-5463ebc667bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:37:34.535695 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc738260-547f-4de3-908c-5463ebc667bd admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:37:34.546204 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1441/2897] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:37:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:37:34.550103 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79db790e-4775-4db3-9d38-3eee3503bc43 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:37:34.560863 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79db790e-4775-4db3-9d38-3eee3503bc43 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:37:34.565716 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41122fcd-3ad4-4cd0-8708-4283501a627c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:37:34.566406 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41122fcd-3ad4-4cd0-8708-4283501a627c admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:34.566720 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41122fcd-3ad4-4cd0-8708-4283501a627c admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:34.567012 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41122fcd-3ad4-4cd0-8708-4283501a627c admin admin] PATH_INFO: `/v3/projects/8f6312bd83ba49bfaeaac8d0a17429e9/users/b187afd415b64da08665d5d2e5310eea/roles/62f19ea272b447ccaab1576cd9bf4fd9` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:34.582574 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41122fcd-3ad4-4cd0-8708-4283501a627c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f6312bd83ba49bfaeaac8d0a17429e9, user_id=b187afd415b64da08665d5d2e5310eea, role_id=62f19ea272b447ccaab1576cd9bf4fd9)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:37:34.585340 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41122fcd-3ad4-4cd0-8708-4283501a627c admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:37:34.685887 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1457/2898] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:37:34 2022] PUT /identity/v3/projects/8f6312bd83ba49bfaeaac8d0a17429e9/users/b187afd415b64da08665d5d2e5310eea/roles/62f19ea272b447ccaab1576cd9bf4fd9 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:37:34.689907 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc738260-547f-4de3-908c-5463ebc667bd admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:37:34.700771 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc738260-547f-4de3-908c-5463ebc667bd admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:37:34.759168 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cf0d82-e7b6-4c4c-9d77-3aaed2b2f894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:37:34.759906 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cf0d82-e7b6-4c4c-9d77-3aaed2b2f894 admin admin] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:34.760030 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cf0d82-e7b6-4c4c-9d77-3aaed2b2f894 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:34.760314 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cf0d82-e7b6-4c4c-9d77-3aaed2b2f894 admin admin] PATH_INFO: `/v3/roles` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:34.761516 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cf0d82-e7b6-4c4c-9d77-3aaed2b2f894 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:37:34.763152 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cf0d82-e7b6-4c4c-9d77-3aaed2b2f894 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:37:34.776343 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1442/2899] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Aug 25 03:37:34 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:37:34.777672 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41122fcd-3ad4-4cd0-8708-4283501a627c admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:37:34.788045 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41122fcd-3ad4-4cd0-8708-4283501a627c admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:37:34.792655 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a0d0ad-ae41-4db2-93dc-c3b547414998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:37:34.793306 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a0d0ad-ae41-4db2-93dc-c3b547414998 admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:34.793587 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a0d0ad-ae41-4db2-93dc-c3b547414998 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:34.793862 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a0d0ad-ae41-4db2-93dc-c3b547414998 admin admin] PATH_INFO: `/v3/projects/8f6312bd83ba49bfaeaac8d0a17429e9/users/b187afd415b64da08665d5d2e5310eea/roles/62f19ea272b447ccaab1576cd9bf4fd9` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:34.795808 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a0d0ad-ae41-4db2-93dc-c3b547414998 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f6312bd83ba49bfaeaac8d0a17429e9, user_id=b187afd415b64da08665d5d2e5310eea, role_id=62f19ea272b447ccaab1576cd9bf4fd9)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:37:34.798470 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a0d0ad-ae41-4db2-93dc-c3b547414998 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:37:34.800705 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:37:34.810201 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1458/2900] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Aug 25 03:37:34 2022] PUT /identity/v3/projects/8f6312bd83ba49bfaeaac8d0a17429e9/users/b187afd415b64da08665d5d2e5310eea/roles/62f19ea272b447ccaab1576cd9bf4fd9 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:37:34.824051 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a877d2b-cb7a-4773-9568-275eaed733fe None None] REQUEST_METHOD: `POST` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:34.824336 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a877d2b-cb7a-4773-9568-275eaed733fe None None] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:34.824608 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a877d2b-cb7a-4773-9568-275eaed733fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:34.852365 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.auth.core [None req-9a877d2b-cb7a-4773-9568-275eaed733fe None None] MFA Rules not processed for user `b187afd415b64da08665d5d2e5310eea`. Rule list: `[]` (Enabled: `True`). {{(pid=81995) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:37:34.853587 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-9a877d2b-cb7a-4773-9568-275eaed733fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:37:34.903249 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:37:34.921451 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1443/2901] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:37:34 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:37:34.933773 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a0d0ad-ae41-4db2-93dc-c3b547414998 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:37:34.935559 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-04a0d0ad-ae41-4db2-93dc-c3b547414998 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:37:34.991544 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a0d0ad-ae41-4db2-93dc-c3b547414998 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:37:34.995749 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caf1116-9783-4b89-96b9-049bd1148e67 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60193ba9e1c14df3bc9eebcc6f28f2d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:37:34.996349 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caf1116-9783-4b89-96b9-049bd1148e67 service cinder] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:34.996632 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caf1116-9783-4b89-96b9-049bd1148e67 service cinder] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:34.996904 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caf1116-9783-4b89-96b9-049bd1148e67 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:35.050618 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caf1116-9783-4b89-96b9-049bd1148e67 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:37:35.052052 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caf1116-9783-4b89-96b9-049bd1148e67 service cinder] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:37:35.061852 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1459/2902] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:37:34 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:37:36.980022 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a877d2b-cb7a-4773-9568-275eaed733fe None None] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:37:36.982050 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-9a877d2b-cb7a-4773-9568-275eaed733fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:37:36.994348 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a877d2b-cb7a-4773-9568-275eaed733fe None None] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:37:36.997984 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2d6e57-4a10-43b5-b8ee-04c2127b95fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60193ba9e1c14df3bc9eebcc6f28f2d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:37:37.000394 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2d6e57-4a10-43b5-b8ee-04c2127b95fc service cinder] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:37.000394 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2d6e57-4a10-43b5-b8ee-04c2127b95fc service cinder] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:37.000394 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2d6e57-4a10-43b5-b8ee-04c2127b95fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:37.056406 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2d6e57-4a10-43b5-b8ee-04c2127b95fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:37:37.057842 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2d6e57-4a10-43b5-b8ee-04c2127b95fc service cinder] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:37:37.066718 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1444/2903] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:37:36 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:37:38.453643 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caf1116-9783-4b89-96b9-049bd1148e67 service cinder] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:37:38.462144 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caf1116-9783-4b89-96b9-049bd1148e67 service cinder] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:37:38.466337 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bf0713-fab1-492f-a859-93a765df3915 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60193ba9e1c14df3bc9eebcc6f28f2d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:37:38.466948 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bf0713-fab1-492f-a859-93a765df3915 service cinder] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:38.467230 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bf0713-fab1-492f-a859-93a765df3915 service cinder] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:38.467505 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bf0713-fab1-492f-a859-93a765df3915 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:38.476524 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bf0713-fab1-492f-a859-93a765df3915 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:37:38.477954 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bf0713-fab1-492f-a859-93a765df3915 service cinder] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:37:38.487941 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1460/2904] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:37:38 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:37:43.177170 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2d6e57-4a10-43b5-b8ee-04c2127b95fc service cinder] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:37:43.239601 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2d6e57-4a10-43b5-b8ee-04c2127b95fc service cinder] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:37:43.244154 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83ec74f-37c4-4735-9123-d5e888bef188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:37:43.244774 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83ec74f-37c4-4735-9123-d5e888bef188 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:43.245050 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83ec74f-37c4-4735-9123-d5e888bef188 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:43.245321 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83ec74f-37c4-4735-9123-d5e888bef188 admin admin] PATH_INFO: `/v3/users/15291fd8a5ca4e7db90d94b5ed2f529a` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:43.246454 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83ec74f-37c4-4735-9123-d5e888bef188 admin admin] RBAC: Authorizing `identity:delete_user(user_id=15291fd8a5ca4e7db90d94b5ed2f529a)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:37:43.248354 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83ec74f-37c4-4735-9123-d5e888bef188 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:37:43.405537 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-f83ec74f-37c4-4735-9123-d5e888bef188 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15291fd8a5ca4e7db90d94b5ed2f529a'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:37:43.478069 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-f83ec74f-37c4-4735-9123-d5e888bef188 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15291fd8a5ca4e7db90d94b5ed2f529a'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:37:43.478789 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-f83ec74f-37c4-4735-9123-d5e888bef188 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15291fd8a5ca4e7db90d94b5ed2f529a'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:37:43.479440 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-f83ec74f-37c4-4735-9123-d5e888bef188 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15291fd8a5ca4e7db90d94b5ed2f529a'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:37:43.491285 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-f83ec74f-37c4-4735-9123-d5e888bef188 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15291fd8a5ca4e7db90d94b5ed2f529a'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:37:43.503575 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1445/2905] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:37:43 2022] DELETE /identity/v3/users/15291fd8a5ca4e7db90d94b5ed2f529a => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:37:43.508169 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30950b2a-a988-43ca-a009-84b03f6023a2 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:43.508666 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30950b2a-a988-43ca-a009-84b03f6023a2 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:43.508808 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30950b2a-a988-43ca-a009-84b03f6023a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:43.527617 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-30950b2a-a988-43ca-a009-84b03f6023a2 None None] MFA Rules not processed for user `e8b708e7077c461bb73c230d7b1825fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:37:43.529439 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-30950b2a-a988-43ca-a009-84b03f6023a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:37:43.586308 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1461/2906] 10.222.0.58 () {58 vars in 958 bytes} [Thu Aug 25 03:37:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:37:43.596692 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83ec74f-37c4-4735-9123-d5e888bef188 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:37:43.598094 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-f83ec74f-37c4-4735-9123-d5e888bef188 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:37:43.608180 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:37:43.676762 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83ec74f-37c4-4735-9123-d5e888bef188 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:37:43.680372 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adc277e-e96f-4a44-96c6-b1063ae92bd4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:37:43.680974 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adc277e-e96f-4a44-96c6-b1063ae92bd4 service neutron] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:43.681507 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adc277e-e96f-4a44-96c6-b1063ae92bd4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:43.681781 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adc277e-e96f-4a44-96c6-b1063ae92bd4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:43.690634 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adc277e-e96f-4a44-96c6-b1063ae92bd4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:37:43.692058 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adc277e-e96f-4a44-96c6-b1063ae92bd4 service neutron] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:37:43.702098 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1446/2907] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:37:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:37:44.120037 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30950b2a-a988-43ca-a009-84b03f6023a2 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:37:44.121676 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-30950b2a-a988-43ca-a009-84b03f6023a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:37:44.136200 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30950b2a-a988-43ca-a009-84b03f6023a2 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:37:44.141673 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c134518-007b-4cea-ac04-a2b9cd4f033f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:37:44.142939 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c134518-007b-4cea-ac04-a2b9cd4f033f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:44.143236 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c134518-007b-4cea-ac04-a2b9cd4f033f admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:44.143699 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c134518-007b-4cea-ac04-a2b9cd4f033f admin admin] PATH_INFO: `/v3/projects/b9124fd8b4394c3b90e6b871eed8380e` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:44.144989 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c134518-007b-4cea-ac04-a2b9cd4f033f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9124fd8b4394c3b90e6b871eed8380e)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:37:44.147035 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c134518-007b-4cea-ac04-a2b9cd4f033f admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:37:44.226148 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:INSERT=1,keystone:SELECT=97,keystone:DELETE=2 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:37:44.239506 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-9c134518-007b-4cea-ac04-a2b9cd4f033f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9124fd8b4394c3b90e6b871eed8380e', 'request_id': 'req-9c134518-007b-4cea-ac04-a2b9cd4f033f'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:37:44.295412 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-9c134518-007b-4cea-ac04-a2b9cd4f033f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9124fd8b4394c3b90e6b871eed8380e', 'request_id': 'req-9c134518-007b-4cea-ac04-a2b9cd4f033f'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:37:44.301214 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-9c134518-007b-4cea-ac04-a2b9cd4f033f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9124fd8b4394c3b90e6b871eed8380e', 'request_id': 'req-9c134518-007b-4cea-ac04-a2b9cd4f033f'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:37:44.314190 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-9c134518-007b-4cea-ac04-a2b9cd4f033f admin admin] The token cache is being invalidate because project b9124fd8b4394c3b90e6b871eed8380e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:37:44.314574 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-9c134518-007b-4cea-ac04-a2b9cd4f033f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:37:44.315242 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-9c134518-007b-4cea-ac04-a2b9cd4f033f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:37:44.316800 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1462/2908] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:37:44 2022] DELETE /identity/v3/projects/b9124fd8b4394c3b90e6b871eed8380e => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:37:44.320856 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adc277e-e96f-4a44-96c6-b1063ae92bd4 service neutron] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:37:44.330793 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adc277e-e96f-4a44-96c6-b1063ae92bd4 service neutron] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:37:44.390935 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7019a64-e383-41b6-860b-7e0f2a42992f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:37:44.391575 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7019a64-e383-41b6-860b-7e0f2a42992f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:44.391857 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7019a64-e383-41b6-860b-7e0f2a42992f admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:44.392135 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7019a64-e383-41b6-860b-7e0f2a42992f admin admin] PATH_INFO: `/v3/users/c239889079434d13bb1d47f9ad240a1a` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:44.393275 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7019a64-e383-41b6-860b-7e0f2a42992f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c239889079434d13bb1d47f9ad240a1a)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:37:44.395258 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7019a64-e383-41b6-860b-7e0f2a42992f admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:37:44.555439 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-f7019a64-e383-41b6-860b-7e0f2a42992f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c239889079434d13bb1d47f9ad240a1a'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:37:44.626818 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-f7019a64-e383-41b6-860b-7e0f2a42992f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c239889079434d13bb1d47f9ad240a1a'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:37:44.627489 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-f7019a64-e383-41b6-860b-7e0f2a42992f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c239889079434d13bb1d47f9ad240a1a'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:37:44.628151 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-f7019a64-e383-41b6-860b-7e0f2a42992f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c239889079434d13bb1d47f9ad240a1a'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:37:44.641801 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-f7019a64-e383-41b6-860b-7e0f2a42992f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c239889079434d13bb1d47f9ad240a1a'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:37:44.654656 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1447/2909] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:37:44 2022] DELETE /identity/v3/users/c239889079434d13bb1d47f9ad240a1a => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:37:45.062941 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c134518-007b-4cea-ac04-a2b9cd4f033f admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:37:45.064503 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-9c134518-007b-4cea-ac04-a2b9cd4f033f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:37:45.079827 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c134518-007b-4cea-ac04-a2b9cd4f033f admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:37:45.138729 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a0d4e2-84a3-494a-80d3-6fd00b0c488f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:37:45.139359 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a0d4e2-84a3-494a-80d3-6fd00b0c488f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:45.139637 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a0d4e2-84a3-494a-80d3-6fd00b0c488f admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:45.139906 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a0d4e2-84a3-494a-80d3-6fd00b0c488f admin admin] PATH_INFO: `/v3/projects/05de7488ec9c413da49551fcf3b09bc3` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:45.141000 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a0d4e2-84a3-494a-80d3-6fd00b0c488f admin admin] RBAC: Authorizing `identity:delete_project(project_id=05de7488ec9c413da49551fcf3b09bc3)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:37:45.142939 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a0d4e2-84a3-494a-80d3-6fd00b0c488f admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:37:45.232068 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-86a0d4e2-84a3-494a-80d3-6fd00b0c488f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05de7488ec9c413da49551fcf3b09bc3', 'request_id': 'req-86a0d4e2-84a3-494a-80d3-6fd00b0c488f'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:37:45.289554 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-86a0d4e2-84a3-494a-80d3-6fd00b0c488f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05de7488ec9c413da49551fcf3b09bc3', 'request_id': 'req-86a0d4e2-84a3-494a-80d3-6fd00b0c488f'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:37:45.295787 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-86a0d4e2-84a3-494a-80d3-6fd00b0c488f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05de7488ec9c413da49551fcf3b09bc3', 'request_id': 'req-86a0d4e2-84a3-494a-80d3-6fd00b0c488f'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:37:45.307975 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-86a0d4e2-84a3-494a-80d3-6fd00b0c488f admin admin] The token cache is being invalidate because project 05de7488ec9c413da49551fcf3b09bc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:37:45.308268 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-86a0d4e2-84a3-494a-80d3-6fd00b0c488f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:37:45.308910 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-86a0d4e2-84a3-494a-80d3-6fd00b0c488f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:37:45.310427 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1463/2910] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:37:45 2022] DELETE /identity/v3/projects/05de7488ec9c413da49551fcf3b09bc3 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:37:45.314531 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7019a64-e383-41b6-860b-7e0f2a42992f admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:37:45.316046 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-f7019a64-e383-41b6-860b-7e0f2a42992f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:37:45.335094 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7019a64-e383-41b6-860b-7e0f2a42992f admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:37:45.340131 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:37:45.396046 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137c1228-998a-4f76-9308-36d3587ef094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:37:45.396677 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137c1228-998a-4f76-9308-36d3587ef094 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:45.396997 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137c1228-998a-4f76-9308-36d3587ef094 admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:45.397289 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137c1228-998a-4f76-9308-36d3587ef094 admin admin] PATH_INFO: `/v3/users/b187afd415b64da08665d5d2e5310eea` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:45.398512 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137c1228-998a-4f76-9308-36d3587ef094 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b187afd415b64da08665d5d2e5310eea)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:37:45.401106 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137c1228-998a-4f76-9308-36d3587ef094 admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:37:45.554304 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-137c1228-998a-4f76-9308-36d3587ef094 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b187afd415b64da08665d5d2e5310eea'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:37:45.615924 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-137c1228-998a-4f76-9308-36d3587ef094 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b187afd415b64da08665d5d2e5310eea'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:37:45.616473 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-137c1228-998a-4f76-9308-36d3587ef094 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b187afd415b64da08665d5d2e5310eea'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:37:45.617092 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-137c1228-998a-4f76-9308-36d3587ef094 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b187afd415b64da08665d5d2e5310eea'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:37:45.630267 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-137c1228-998a-4f76-9308-36d3587ef094 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b187afd415b64da08665d5d2e5310eea'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:37:45.644870 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1448/2911] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:37:45 2022] DELETE /identity/v3/users/b187afd415b64da08665d5d2e5310eea => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:37:46.057125 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a0d4e2-84a3-494a-80d3-6fd00b0c488f admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:37:46.058587 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-86a0d4e2-84a3-494a-80d3-6fd00b0c488f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:37:46.073908 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a0d4e2-84a3-494a-80d3-6fd00b0c488f admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:37:46.118911 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:37:46.131173 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabbc178-fab6-4bbd-97b9-d2249e1a0a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:37:46.131809 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabbc178-fab6-4bbd-97b9-d2249e1a0a6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:37:46.132090 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabbc178-fab6-4bbd-97b9-d2249e1a0a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:37:46.132362 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabbc178-fab6-4bbd-97b9-d2249e1a0a6d admin admin] PATH_INFO: `/v3/projects/8f6312bd83ba49bfaeaac8d0a17429e9` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:37:46.133468 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabbc178-fab6-4bbd-97b9-d2249e1a0a6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f6312bd83ba49bfaeaac8d0a17429e9)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:37:46.135454 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabbc178-fab6-4bbd-97b9-d2249e1a0a6d admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:37:46.255324 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-eabbc178-fab6-4bbd-97b9-d2249e1a0a6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f6312bd83ba49bfaeaac8d0a17429e9', 'request_id': 'req-eabbc178-fab6-4bbd-97b9-d2249e1a0a6d'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:37:46.335298 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-eabbc178-fab6-4bbd-97b9-d2249e1a0a6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f6312bd83ba49bfaeaac8d0a17429e9', 'request_id': 'req-eabbc178-fab6-4bbd-97b9-d2249e1a0a6d'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:37:46.341364 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-eabbc178-fab6-4bbd-97b9-d2249e1a0a6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f6312bd83ba49bfaeaac8d0a17429e9', 'request_id': 'req-eabbc178-fab6-4bbd-97b9-d2249e1a0a6d'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:37:46.353143 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-eabbc178-fab6-4bbd-97b9-d2249e1a0a6d admin admin] The token cache is being invalidate because project 8f6312bd83ba49bfaeaac8d0a17429e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:37:46.353431 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-eabbc178-fab6-4bbd-97b9-d2249e1a0a6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:37:46.354054 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-eabbc178-fab6-4bbd-97b9-d2249e1a0a6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:37:46.355561 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1464/2912] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:37:46 2022] DELETE /identity/v3/projects/8f6312bd83ba49bfaeaac8d0a17429e9 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:37:55.640869 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=49,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:37:56.349600 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=29,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:38:09.225297 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137c1228-998a-4f76-9308-36d3587ef094 admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:38:09.226741 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-137c1228-998a-4f76-9308-36d3587ef094 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:38:09.294596 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137c1228-998a-4f76-9308-36d3587ef094 admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:38:09.352132 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14ea35e-0c60-44cd-87c8-a9dac370726e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b692edd3e8c41289cc42e1afb4a6dd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:38:09.352783 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14ea35e-0c60-44cd-87c8-a9dac370726e service placement] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:38:09.353074 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14ea35e-0c60-44cd-87c8-a9dac370726e service placement] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:38:09.353473 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14ea35e-0c60-44cd-87c8-a9dac370726e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:38:09.355237 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-b14ea35e-0c60-44cd-87c8-a9dac370726e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:38:09.367161 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14ea35e-0c60-44cd-87c8-a9dac370726e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:38:09.368738 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14ea35e-0c60-44cd-87c8-a9dac370726e service placement] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:38:09.378586 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1449/2913] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:38:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:38:19.374254 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=58 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:39:05.374132 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabbc178-fab6-4bbd-97b9-d2249e1a0a6d admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:39:05.375659 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-eabbc178-fab6-4bbd-97b9-d2249e1a0a6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:39:05.434390 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabbc178-fab6-4bbd-97b9-d2249e1a0a6d admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:39:05.437968 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412be2fc-52b1-4a87-87e9-2215087daa0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:39:05.438598 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412be2fc-52b1-4a87-87e9-2215087daa0f service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:39:05.438881 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412be2fc-52b1-4a87-87e9-2215087daa0f service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:39:05.439156 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412be2fc-52b1-4a87-87e9-2215087daa0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:39:05.440866 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-412be2fc-52b1-4a87-87e9-2215087daa0f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:39:05.527233 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412be2fc-52b1-4a87-87e9-2215087daa0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:39:05.528665 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412be2fc-52b1-4a87-87e9-2215087daa0f service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:39:05.548897 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1465/2914] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:39:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:39:15.546293 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=66 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:39:18.533802 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14ea35e-0c60-44cd-87c8-a9dac370726e service placement] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:39:18.543928 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14ea35e-0c60-44cd-87c8-a9dac370726e service placement] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:39:18.547484 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8706e403-57ba-4624-9a55-5a8933db080c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:39:18.548078 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8706e403-57ba-4624-9a55-5a8933db080c service neutron] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:39:18.548358 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8706e403-57ba-4624-9a55-5a8933db080c service neutron] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:39:18.548632 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8706e403-57ba-4624-9a55-5a8933db080c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:39:18.550436 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-8706e403-57ba-4624-9a55-5a8933db080c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:39:18.563959 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8706e403-57ba-4624-9a55-5a8933db080c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:39:18.565377 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8706e403-57ba-4624-9a55-5a8933db080c service neutron] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:39:18.576664 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1450/2915] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:39:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:39:18.749327 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412be2fc-52b1-4a87-87e9-2215087daa0f service neutron] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:39:18.758595 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412be2fc-52b1-4a87-87e9-2215087daa0f service neutron] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:39:18.762272 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2bca6a-d572-44f8-8404-f0d0e40da184 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:39:18.768357 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2bca6a-d572-44f8-8404-f0d0e40da184 service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:39:18.768357 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2bca6a-d572-44f8-8404-f0d0e40da184 service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:39:18.768357 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2bca6a-d572-44f8-8404-f0d0e40da184 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:39:18.768357 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-ce2bca6a-d572-44f8-8404-f0d0e40da184 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:39:18.775266 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2bca6a-d572-44f8-8404-f0d0e40da184 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:39:18.776680 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2bca6a-d572-44f8-8404-f0d0e40da184 service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:39:18.784633 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1466/2916] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:39:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 25 03:39:28.571614 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=6 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:39:28.781054 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=6 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:40:13.946656 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8706e403-57ba-4624-9a55-5a8933db080c service neutron] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:40:13.955828 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8706e403-57ba-4624-9a55-5a8933db080c service neutron] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:40:13.959413 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feaed15-7a6f-4c4a-9454-7b0309183de5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:40:13.960003 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feaed15-7a6f-4c4a-9454-7b0309183de5 service neutron] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:40:13.960284 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feaed15-7a6f-4c4a-9454-7b0309183de5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:40:13.960558 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feaed15-7a6f-4c4a-9454-7b0309183de5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:40:13.962244 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-0feaed15-7a6f-4c4a-9454-7b0309183de5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:40:13.971744 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0feaed15-7a6f-4c4a-9454-7b0309183de5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:40:13.973160 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0feaed15-7a6f-4c4a-9454-7b0309183de5 service neutron] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:40:13.981051 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1451/2917] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:40:13 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:40:23.977510 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=6 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:41:10.475810 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2bca6a-d572-44f8-8404-f0d0e40da184 service neutron] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:41:10.477228 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-ce2bca6a-d572-44f8-8404-f0d0e40da184 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:41:10.487661 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2bca6a-d572-44f8-8404-f0d0e40da184 service neutron] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:41:10.491258 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82fb1c5-8424-4361-8784-b9df39589281 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:41:10.491878 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82fb1c5-8424-4361-8784-b9df39589281 service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:41:10.492159 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82fb1c5-8424-4361-8784-b9df39589281 service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:41:10.492433 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82fb1c5-8424-4361-8784-b9df39589281 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:41:10.494075 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-d82fb1c5-8424-4361-8784-b9df39589281 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:41:10.555327 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82fb1c5-8424-4361-8784-b9df39589281 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:41:10.557015 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82fb1c5-8424-4361-8784-b9df39589281 service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:41:10.565982 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1467/2918] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:41:10 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:41:15.028303 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feaed15-7a6f-4c4a-9454-7b0309183de5 service neutron] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:41:15.035316 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-0feaed15-7a6f-4c4a-9454-7b0309183de5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:41:15.046498 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feaed15-7a6f-4c4a-9454-7b0309183de5 service neutron] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:41:15.052313 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a668200c-fc7a-4ce7-9a9d-123c46ac1651 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cd171bf4df94f8fab397e42835feeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:41:15.052994 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a668200c-fc7a-4ce7-9a9d-123c46ac1651 service nova] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:41:15.053276 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a668200c-fc7a-4ce7-9a9d-123c46ac1651 service nova] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:41:15.053555 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a668200c-fc7a-4ce7-9a9d-123c46ac1651 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:41:15.055252 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-a668200c-fc7a-4ce7-9a9d-123c46ac1651 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:41:15.067617 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a668200c-fc7a-4ce7-9a9d-123c46ac1651 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:41:15.069173 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a668200c-fc7a-4ce7-9a9d-123c46ac1651 service nova] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:41:15.080094 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1452/2919] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:41:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:41:17.208766 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82fb1c5-8424-4361-8784-b9df39589281 service neutron] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:41:17.217411 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82fb1c5-8424-4361-8784-b9df39589281 service neutron] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:41:17.220954 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30363972-2254-4cb0-b32c-1ce31d4a5dff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a1a7f091774842907fd08de15e8058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:41:17.221546 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30363972-2254-4cb0-b32c-1ce31d4a5dff service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:41:17.221822 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30363972-2254-4cb0-b32c-1ce31d4a5dff service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:41:17.222150 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30363972-2254-4cb0-b32c-1ce31d4a5dff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:41:17.224124 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-30363972-2254-4cb0-b32c-1ce31d4a5dff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:41:17.287732 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30363972-2254-4cb0-b32c-1ce31d4a5dff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:41:17.289182 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30363972-2254-4cb0-b32c-1ce31d4a5dff service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:41:17.299113 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1468/2920] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:41:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:41:24.237511 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a668200c-fc7a-4ce7-9a9d-123c46ac1651 service nova] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:41:24.239043 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-a668200c-fc7a-4ce7-9a9d-123c46ac1651 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:41:24.253667 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a668200c-fc7a-4ce7-9a9d-123c46ac1651 service nova] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:41:24.258559 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414e205c-5e3e-4d1a-87dc-bf8c0593c33a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:41:24.259182 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414e205c-5e3e-4d1a-87dc-bf8c0593c33a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:41:24.259463 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414e205c-5e3e-4d1a-87dc-bf8c0593c33a admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:41:24.259735 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414e205c-5e3e-4d1a-87dc-bf8c0593c33a admin admin] PATH_INFO: `/v3/users/9f32ba4c897c4003ba27ebc78c6d05a1` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:41:24.260841 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414e205c-5e3e-4d1a-87dc-bf8c0593c33a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f32ba4c897c4003ba27ebc78c6d05a1)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:41:24.262813 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414e205c-5e3e-4d1a-87dc-bf8c0593c33a admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:41:24.415290 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-414e205c-5e3e-4d1a-87dc-bf8c0593c33a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f32ba4c897c4003ba27ebc78c6d05a1'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:41:24.482078 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-414e205c-5e3e-4d1a-87dc-bf8c0593c33a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f32ba4c897c4003ba27ebc78c6d05a1'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:41:24.482715 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-414e205c-5e3e-4d1a-87dc-bf8c0593c33a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f32ba4c897c4003ba27ebc78c6d05a1'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:41:24.483328 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-414e205c-5e3e-4d1a-87dc-bf8c0593c33a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f32ba4c897c4003ba27ebc78c6d05a1'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:41:24.495518 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-414e205c-5e3e-4d1a-87dc-bf8c0593c33a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f32ba4c897c4003ba27ebc78c6d05a1'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:41:24.507961 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1453/2921] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:41:24 2022] DELETE /identity/v3/users/9f32ba4c897c4003ba27ebc78c6d05a1 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:41:24.760705 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30363972-2254-4cb0-b32c-1ce31d4a5dff service neutron] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:41:24.768909 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30363972-2254-4cb0-b32c-1ce31d4a5dff service neutron] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:41:24.832572 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8e02a1-e08f-4823-bdcd-24dbf1b5368a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:41:24.833206 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8e02a1-e08f-4823-bdcd-24dbf1b5368a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:41:24.833489 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8e02a1-e08f-4823-bdcd-24dbf1b5368a admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:41:24.833765 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8e02a1-e08f-4823-bdcd-24dbf1b5368a admin admin] PATH_INFO: `/v3/projects/d34a968ff59e43e796b4b48b35374bc7` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:41:24.835028 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8e02a1-e08f-4823-bdcd-24dbf1b5368a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d34a968ff59e43e796b4b48b35374bc7)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:41:24.836881 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8e02a1-e08f-4823-bdcd-24dbf1b5368a admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:41:24.947213 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:41:24.956466 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-2b8e02a1-e08f-4823-bdcd-24dbf1b5368a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd34a968ff59e43e796b4b48b35374bc7', 'request_id': 'req-2b8e02a1-e08f-4823-bdcd-24dbf1b5368a'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:41:25.019341 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-2b8e02a1-e08f-4823-bdcd-24dbf1b5368a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd34a968ff59e43e796b4b48b35374bc7', 'request_id': 'req-2b8e02a1-e08f-4823-bdcd-24dbf1b5368a'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:41:25.025009 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-2b8e02a1-e08f-4823-bdcd-24dbf1b5368a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd34a968ff59e43e796b4b48b35374bc7', 'request_id': 'req-2b8e02a1-e08f-4823-bdcd-24dbf1b5368a'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:41:25.036963 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-2b8e02a1-e08f-4823-bdcd-24dbf1b5368a admin admin] The token cache is being invalidate because project d34a968ff59e43e796b4b48b35374bc7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:41:25.037332 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-2b8e02a1-e08f-4823-bdcd-24dbf1b5368a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:41:25.037968 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-2b8e02a1-e08f-4823-bdcd-24dbf1b5368a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:41:25.039475 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1469/2922] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:41:24 2022] DELETE /identity/v3/projects/d34a968ff59e43e796b4b48b35374bc7 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:41:25.043482 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414e205c-5e3e-4d1a-87dc-bf8c0593c33a admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:41:25.044976 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-414e205c-5e3e-4d1a-87dc-bf8c0593c33a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:41:25.058957 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414e205c-5e3e-4d1a-87dc-bf8c0593c33a admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:41:25.115982 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce4c386-0b9c-4f3a-ad4e-9faa41812d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:41:25.116604 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce4c386-0b9c-4f3a-ad4e-9faa41812d9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:41:25.116887 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce4c386-0b9c-4f3a-ad4e-9faa41812d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:41:25.117162 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce4c386-0b9c-4f3a-ad4e-9faa41812d9d admin admin] PATH_INFO: `/v3/users/05c23669d16a4f6fa276cf1997dc49f4` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:41:25.118315 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce4c386-0b9c-4f3a-ad4e-9faa41812d9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=05c23669d16a4f6fa276cf1997dc49f4)` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:41:25.120240 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce4c386-0b9c-4f3a-ad4e-9faa41812d9d admin admin] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:41:25.250120 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-fce4c386-0b9c-4f3a-ad4e-9faa41812d9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05c23669d16a4f6fa276cf1997dc49f4'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:41:25.304726 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-fce4c386-0b9c-4f3a-ad4e-9faa41812d9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05c23669d16a4f6fa276cf1997dc49f4'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:41:25.305351 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-fce4c386-0b9c-4f3a-ad4e-9faa41812d9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05c23669d16a4f6fa276cf1997dc49f4'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:41:25.306209 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-fce4c386-0b9c-4f3a-ad4e-9faa41812d9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05c23669d16a4f6fa276cf1997dc49f4'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:41:25.311576 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:41:25.318668 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.notifications [None req-fce4c386-0b9c-4f3a-ad4e-9faa41812d9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05c23669d16a4f6fa276cf1997dc49f4'} {{(pid=81995) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:41:25.331050 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1454/2923] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Aug 25 03:41:25 2022] DELETE /identity/v3/users/05c23669d16a4f6fa276cf1997dc49f4 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:41:25.564199 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8e02a1-e08f-4823-bdcd-24dbf1b5368a admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:41:25.566030 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-2b8e02a1-e08f-4823-bdcd-24dbf1b5368a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:41:25.581864 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8e02a1-e08f-4823-bdcd-24dbf1b5368a admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:41:25.642877 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543ca056-a957-42ad-9316-12047ed3f47d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8b708e7077c461bb73c230d7b1825fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b981090d110a4dd99327dbd654b9daa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:41:25.643536 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543ca056-a957-42ad-9316-12047ed3f47d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:41:25.643894 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543ca056-a957-42ad-9316-12047ed3f47d admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:41:25.644177 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543ca056-a957-42ad-9316-12047ed3f47d admin admin] PATH_INFO: `/v3/projects/51a8b420242541a08fce3ca9da4e8559` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:41:25.645301 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543ca056-a957-42ad-9316-12047ed3f47d admin admin] RBAC: Authorizing `identity:delete_project(project_id=51a8b420242541a08fce3ca9da4e8559)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:41:25.647316 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543ca056-a957-42ad-9316-12047ed3f47d admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:41:25.734257 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-543ca056-a957-42ad-9316-12047ed3f47d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51a8b420242541a08fce3ca9da4e8559', 'request_id': 'req-543ca056-a957-42ad-9316-12047ed3f47d'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:41:25.790329 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-543ca056-a957-42ad-9316-12047ed3f47d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51a8b420242541a08fce3ca9da4e8559', 'request_id': 'req-543ca056-a957-42ad-9316-12047ed3f47d'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:41:25.796766 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-543ca056-a957-42ad-9316-12047ed3f47d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51a8b420242541a08fce3ca9da4e8559', 'request_id': 'req-543ca056-a957-42ad-9316-12047ed3f47d'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:41:25.808102 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-543ca056-a957-42ad-9316-12047ed3f47d admin admin] The token cache is being invalidate because project 51a8b420242541a08fce3ca9da4e8559 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:41:25.808338 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-543ca056-a957-42ad-9316-12047ed3f47d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:41:25.808968 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-543ca056-a957-42ad-9316-12047ed3f47d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:41:25.810482 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1470/2924] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Aug 25 03:41:25 2022] DELETE /identity/v3/projects/51a8b420242541a08fce3ca9da4e8559 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:41:35.413464 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=9,keystone:DELETE=1 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:41:35.804818 dv-cinder-851640-5-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:DELETE=12,keystone:SELECT=62,keystone:INSERT=2 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:43:17.199019 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce4c386-0b9c-4f3a-ad4e-9faa41812d9d admin admin] Authenticating user token {{(pid=81995) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:43:17.200445 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-fce4c386-0b9c-4f3a-ad4e-9faa41812d9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:43:17.269778 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce4c386-0b9c-4f3a-ad4e-9faa41812d9d admin admin] Validating token access rules against request {{(pid=81995) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:43:17.327642 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3723f2df-2184-4aab-b47b-28d563c4ed66 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b692edd3e8c41289cc42e1afb4a6dd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fd4be43faec4e4191e0e4ec4a4588c3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81995) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:43:17.328246 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3723f2df-2184-4aab-b47b-28d563c4ed66 service placement] REQUEST_METHOD: `GET` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:43:17.328529 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3723f2df-2184-4aab-b47b-28d563c4ed66 service placement] SCRIPT_NAME: `/identity` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:43:17.328804 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3723f2df-2184-4aab-b47b-28d563c4ed66 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81995) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:43:17.330569 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.fernet_utils [None req-3723f2df-2184-4aab-b47b-28d563c4ed66 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81995) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:43:17.342479 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3723f2df-2184-4aab-b47b-28d563c4ed66 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:43:17.344059 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3723f2df-2184-4aab-b47b-28d563c4ed66 service placement] RBAC: Authorization granted {{(pid=81995) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:43:17.354264 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: [pid: 81995|app: 0|req: 1455/2925] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Aug 25 03:43:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:43:27.349779 dv-cinder-851640-5-iscsi devstack@keystone.service[81995]: DEBUG dbcounter [-] [81995] Writing DB stats keystone:SELECT=58 {{(pid=81995) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}